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ABSTRACT 

This  proceedings  of  the  1993  convention  of  the 
Association  for  Educational  Communications  and  Technology  (AECT) 
contains  70  papers  and  1  symposium.  Topics  addressed  by  the  papers 
include  computer-based  mapping  for  curriculum  development; 
educational  technology  equipped  classrooms;  individual 
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education;  computer  simulations;  cooperative  learning;  Channel  One; 
groupware;  restructuring  library  media  centers;  media  and  technology 
competencies;   research  methodologi es ;  design  of  computer-based 
instruction;   learner-computer  interaction;  constructivism;  and 
library  instruction.  The  symposium  topic  was  critical  theory, 
cultural  analysis,  and  ethical  aspects  of  the  field.  This  volume  also 
includes  a  list  of  Research  and  Theory  Division  officers;  a  list  of 
the  reviewers  of  the  conference  papers;  a  list  of  proceedings  from 
1979-92  together  with  their  ERIC  document  numbers;  a  description  of 
AECT  and  a  membership  enrollment  form;  and  author  and  descriptor 
indexes .  (MES) 
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PREFACE 


For  the  fifteenth  year,  the  Research  and  Theory  Division  of  the  Association  for  Educational 
Communications  and  Technology  (AECT)  is  sponsoring  the  publications  of  these  Proceedings. 
Papers  published  in  this  volume  were  presented  at  the  rational  AECT  Convention  in  New  Orleans, 
LA*  A  limited  quantity  of  this  volume  were  printed  and  sold  It  is  also  available  on  microfiche 
through  the  Educational  Resources  Information  Clearinghouse  (ERIC)  system. 

REFEREEING  PROCESS:  All  research  papers  selected  for  presentation  at  the  AECT 
convention  and  included  in  this  EcQ£££dui£S  were  subjected  to  a  rigorous  blind  reviewing  process. 
All  references  to  author  were  removed  from  proposals  before  they  were  submitted  to  referees  for 
review.  Approximately  fifty  percent  of  the  manuscripts  submitted  for  consideration  were  selected 
for  presentation  at  the  Convention  and  for  Publication  in  these  ECQCGfidillgfr  The  papers  contained 
in  this  document  represent  some  of  the  most  current  thinking  in  educational  communications  and 
technology. 

For  the  first  time,  a  selected  number  of  development  papers,  sponsored  by  the  Division  for 
Instructional  Development  (DID),  are  included  in  this  Proceedings.  The  most  important 
instructional  development  papers  were  selected  by  the  DID  program  chairs  for  publication. 

This  volume  contains  two  indexes.  The  first  is  an  author  index;  the  second  is  a  descriptor 
index.  The  index  for  volumes  1-6  (1979-84)  is  included  in  the  1086  Proceedings,  and  the  index  for 
volumes  7-10  is  in  the  1988  Proceeding. 
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Editor 
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What  Is  AECT? 

AECT  Is  tho  onV  national,  professional  association  dedicated 
to  the  Improvement  of  Iratructlon/fivough  the  effective  use  of 
media  and  technology.  AECT  assists  Its  members  in  using 
technology  in  their  Jobs  and  l  o  enhance  the  learning  process. 


Who  belongs  to  AECT? 

(J  Media  Specialists 

□  Educators 

□  Librarians 

□  Instructional  designers 

(J  Corporate/military  trainers 

□  Learning  resource  specialists 

□  Curriculum  developers 

□  Television  producers  and  directors 

□  Communications  specialists 

□  Education  administrators 

□  Others  who  require  expertise  In  Instructional  technology 


What  are  AECT  members  Involved  In? 


Hypermedia 

□ 

Interactive  Video 

□ 

CD-ROM 

□ 

ca 

□ 

Teleconferencing 

□ 

Film  &  video  utilization 

□ 

Telecommunications 

u 

Computer  software  &  hardware 

IJ 

Projection/presentation  products 

u 

Intelligent  tutoring  systems 

u 

Videodiscs 

u 

Distance  learning 

u 

And  more! 

AECT's  publications,  convention,  trade  show,  and 
conferences  present  the  leading  edge  on  research  and 
practical  applications  for  these  and  other  technologies. 


AECT  History 

Unlike  some  other  special-interest  technology  organizations. 
AECT  has  a  long  history,  with  over  70  years  In  educational 
technology.  We've  grown  up  with  technology,  advocating 
Integration  Into  education  from  Alms  to  Integration  Into 
education  from  Alms  to  overheads  to  interactive  video  and 
hypermedia. 


AECT  began  as  the  Department  of  Visual  instruction  at  the 
National  Education  Association  in  1923,  in  the  days  when  visual 
aids  consisted  of  films  and  slides,  in  1947,  as  educators  were 
adapt^g  technology  used  to  +raln  World  War  II  service 
personnel  for  the  classroom,  the  name  of  the  organization 
became  the  Department  of  Visual  instruction  (DAVI).  Twelve 
years  later,  DAVI  became  an  affiliate  of  the  NEA  and  finally  the 
autonomous  association.  AECT.  in  1974. 

Today.  AECT  keeps  an  eye  on  the  future  of  instructional 
technology  while  assisting  educators  with  the  changes  and 
challenges  that  face  them  now.  AECT  members,  now 
numbering  4,500  are  professionals  devoted  to  quality 
education. 

Tney  care  about  doing  their  Jobs  better  and  want  to  embrace 
new  methods,  new  equipment,  and  new  techniques  that  assist 
learning. 

Membership  in  AECT  Increases  your  effectiveness,  your 
expertise,  and  your  skills.  These  qualities  In  turn  enhance  your 
professional  Image  and  earning  potential. 


AECT  Affiliates 

AECT  has  47  state  and  16  regional  and  national  affiliates,  and 
has  recently  established  several  chapters  surrounding  major 
universities  and  metropolitan  areas.  These  affiliated 
organizations  add  a  localized  dimension  to  your  AECT 
membership  and  allow  for  more  interaction  among  your 
colleagues.  For  more  details  on  chapters  and  affiliates  in  your 
area,  contact  the  AECT  National  Office. 


AECT  National  Convention  and  InCITE 
Exposition 

Each  year.  AECT  brings  Kv  speakers  to  exciting  locations,  and 
presents  over  300  sessions  anJ  special  events  to  provide  the 
best  training  available  In  the  use  of  media  In  education  and 
instruction.  The  convention  features  the  InCITE  Exposition,  the 
first  trade  show  created  exclusively  for  instructional 
technology  products.  At  InCITE,  you'll  see  computers,  learning 
systems,  software,  interactive  multimedia,  audiovisual 
products,  films  &  videotapes,  projectors  and  presentation 
products,  video  equipment,  accessories,  and  more.  The 
Convention  offers  tracks  of  sessions  focusing  on  specific 
Interest  areas  surrounding  AECPs  nine  Divisions  and  other 
special  Interests.  The  convention  has  featured  a  Hypermedia 
Strand  and  a  Total  Quality  Management  Track.  In  addition, 
intensive  fun  and  half-day  workshops  are  offered  for  in-depth 
training  on  the  latest  technology  applications  for  education. 


Research  and  Theory  Division  (RTD)  improves  the 

design,  execution,  utilization,  evaluation,  and  dissemination  of 
educational  technology  research  and  theory;  advises 
educators  on  using  research  results. 
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□□  Yourself  a  Favor . 


JOIN  AECT  TODAY! 

AECT  has  nine  spca-l-intcrcsl  divisions  to  meet  your  needs  and  address  your  special  interests  within  the  field  of  educational  technology  and  media  Each  division 
publishes  ils  own  newsletter  and  is  devoted  lo  research,  communications  and  publication  of  findings  within  its  specially.  In  addition,  most  divisions  sponsor  an 
awards  program  Your  AECT  membership  includes  one  division,  and  you  may  join  as  many  divisions  as  you  like  lor  $10  each. 


Media  Design  and  Production  Division  (MPPD)  iocuses  on  enhancing  media 
DimliicliOn  skills  and  provides  a  fcrum  for  inleraction  among  researchers, 
commercial  and  noncommercial  media  producers,  and  educators. 

Industrial  Training  and  Education  Division  (ITED)  is  involved  with  designing, 
pl.iiiniiiy  jihJ  managing  training  programs,  and  works  to  promote  maximum 
iihii/alion  ol  edncahonal  techniques  and  media  thai  are  effective  in  practical  use 

Division  of  Interactive  Systems  and  Computers  (DISC)  is  concerned  with 
ihe  design,  development,  delivery,  evaluation  and  migration  ol  interactive  media 
ami  cumpnlui  based  applications  in  education  and  training  The  Division  promotes 
the  networking  of  ils  members  lo  lacititate  sharing  ol  expertise  and  mteresls. 

Division  of  Instructional  Development  (DID)  studies,  evaluates,  and  refines 
design  processes,  crcales  new  models  of  instructional  development,  disseminates 
tindmg:,.  and  promotes  academic  programs 

International  Division  (INTL)  encourages  practice  and  research  m  educational 
communication  and  dislance  education  tor  social  and  economic  development  across 
national  and  ctitlinal  tines  It  promotes  international  exchange  and  sharing  ol 
information,  and  enhances  professional  and  personal  relationships  between  present 
.nut  tuiure  international  leaders 


Division  of  Educational  Media  Management  (DEMM)  creates  guidelines  on 
educational  media  management  and  sponsors  prog»3nis  lo  enhance  media 
management  skills.  The  Division  strives  to  resolve  common  problems  within  the 
industry  and  design  practical  solutions. 

Research  and  Hieory  Division  (RTD)  facilitates  the  design,  execution, 
ulilization.  evaluation,  and  dissemination  of  educational  technology  reseaich, 
promotes  applied  and  theoretical  research  on  Ihe  use  ot  educational  lechnology, 
and  encourages  the  use  of  multiple  research  paradigms  in  examining  issues  related 
to  technology  in  instruction 

Division  of  School  Media  Specialists  (DSMS)  promotes  communications 
among  school  media  personnel  who  share  a  common  concern  in  ll»e  development, 
implementation,  and  evaluation  of  school  media  programs,  and  strives  to  increase 
teaming  and  improve  instruction  in  Ihe  school  setting  through  Ihe  ultetion  of 
educational  media  and  technology 

Division  of  Telecommunications  (DOT)  improves  instruction  thiough  the  use 
ol  letecommunicalions  including  television,  radio,  video,  and  audio  devices  The 
Division  is  concerned  with  design,  pioduclion,  ulili/ahon.  and  evaluation  ol 
instructional  telecommunications  materials  and  equipment,  as  well  as  the  training 
and  qualifications  ot  peisonnel 


MEMBERSHIP  ENROLLMENT  FORM 


ERLC 


Lnroll  me  for  the  membership  calegory.  division(s),  and  options  f  have  selected  below  I  understand  that  my  membership  will  extend  for  12 
months  trow  the  date  my  payment  is  received 


i  I  Regular  Membership.  $65  lee  includes  $12  lor  krJi  f rends  m\  one  tree 
division 

I  i  International  Regular  Membership,  $77  (surface  mail) 
I  i  International  Regular  Membership,  $107  (air  mail) 
I  l  Comprehensive  Membership,  $95  Fee  includes  $12  for  lechhends.  $30 
.  lor  Educational  Jcchnulogy  Research  &  Development,  and  one  free  division 
I  I  International  Comprehensive  Membership,  $115  (suilace  mail) 
I  I  International  Comprehensive  Membership,  $145  (air  marl) 
I  I  Student  Membership,  $26*  fee  includes  $12  fortecftfamfcand  one  free 
division 

I  i  International  Student  Membership,  $38*  (surface  mail) 

I  :  Comprehensive  Student  Membership,  $46*  Fee  includes  $12  lor 

fechfrends.  $20  for  [durational  Technology  Research  &  Development,  and  one 

tree  division 

I  I  International  Comprehensive  Student  Membership,  $66*  (surface  mail) 

'  i  Corporate  Membership,  $390  rec  includes  $12  for  fechftends  and  one 
lien  division 


*  Nulr  MihNiI  .i|i|iln.aiila.  mii',1  lie  enrolled  m  lotmal  coursework  m  ihe 
cdiit.iluiiM!  let  iiiiiiiuqy  f u-hl  mi  ,rl  least  a  half  lime  bases  to  quality  Inr  this 
•i|ieu,il  ule  Applicant;,  mii-.l  unhide  Hit!  name  ol  Ihe  university  and  tiead  of 
Ihe  ileiuilmeiil  Im  ventiLiimn 

Name  ul  UmveiMly  .  


beiktflminil  tle.id . 


-  Phone . 


Division  Memberships  Please  choose  your  one  free  division  from  list  ahove 

riDlSC     tlDOT      I.IRID      r  ]  DID      I  ]  INTL 

MlTED     t  J  DEMM  PI  MDPD   riDSMS  $  hREE 

Options  ($10  each  additional  Divisional  affiliation) 

n  disc   n  dot    nRTD    noiD  ijintl 

r  1  ITED     r  1  DEMM  LI  MDPD  ( ]  DSMS    Total  extra  divisions  $  


Payment 

T 1  My  check  or  money  order  is  enclosed 
f  t  Charge  this  order  to  my  credit  card 
f  I  Mastercard     (1  Visa 


Tolal  Membership  $ . 


Card  # . 


.  Expires . 


Signature . 


I  )  Please  bill  me 

Name,  Title  

Irishluhon  

Address  


.  Phone . 


City.  Slate.  Zip. 


Malt  this  form  to  AECT,  1025  Vermont  Ave.,  NW,  Washington,  DC  20005, 
or  Fai  (202)  347-7131  II  ptylno  ty  credit  cartf. 


Ihn's  to  tttt>  Association  loi  i  dtiuhonat  Commt motions  and  Technology  may  be  deductible  by  members  lor  federal  income  tax  purposes  as  ordinary  and  necessary 
tntsmnH  wfHmw:*  Dues  and  tfills  are  not  deductible  ctwitable  contributions  Please  consult  your  lax  advisor  for  assistancem  your  specific  situation 
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This  article  describes  the  results  of  a  three-month  experiment  in  the 
use  of  computer-based  semantic  networks  for  curriculum  development.. 
A  team  of  doctoral  and  master's  degree  students  developed  a  1200- 
item  computer  data  base  representing  a  tentative  "domain  of 
competency"  for  a  proposed  MA  degree  in  Workforce  Education  and 
Lifelong  Learning  (WELL). 

The  team  gathered  descriptions  of  knowledge  that  might  be  relevant  to 
the  proposed  degree  from  state  and  national  reports,  existing  course 
syllabi,  textbooks,  and  interviews  with  subject  matter  experts.  Using 
SernNet,  a  Macintosh-based  program  for  constructing  and  analyzing 
semantic  networks,  they  explored  methods  for  organizing  these 
descriptions  as  a  "map"  of  related  skills  and  ideas  that  would  in  turn 
serve  as  a  framework  for  a  WELL  curriculum.  The  team  explored 
various  methods  for  using  domain  maps  to  define  course  content,  to 
recommend  learning  activities,  and  to  provide  academic  counseling  to 
prospective  WELL  students.  Among  the  innovative  methods  for 
conducting  this  experiment  in  curriculum  development  was  the  use  of 
computer  software  for  synchronous  conferencing  which  allowed  team 
members  to  collaborate  in  evaluating  and  integrating  domain  maps. 


Curriculum  development  is  a  im\jor  concern  at  all  levels  of  education  since  it 
encompasses  the  goals  of  learning,  the  scope  and  sequence  of  subject  matter  content, 
the  interaction  of  the  learners  with  the  content,  and  methods  for  assessing  the 
outcomes.  In  recent  years,  curriculum  development  has  become  a  much  more 
complex  task  because  of  the  explosion  of  human  knowledge,  the  increased  variety  of 
skills  required  for  successful  employment  and  citizenship,  and  the  greater 
fractiousnesH  of  political  debate  over  content  and  methods. 

A  steady  stream  of  widely-cited  curriculum  theorists  (Tyler,  IDfiO;  Tnbn, 
11108;  Hartley,  1908;  Tanner  and  Tanner,  1980;  Doll,  1989)  have  proponed  criteria 
that  taught  guide  decision  making  in  the  curriculum  planning  process.  However, 
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curriculum  development  is  hardly  an  objective  or  purely  scientific  enterprise  that 
follows  universal  principles  or  a  predetermined  planning  process. 

Decision  making  in  curriculum  development  is  a  function  of  tradition, 
historical  and  philosophical  perspectives,  social  and  cultural  influences,  and  the 
political  exigencies  of  the  day.  While  all  of  these  are  important  to  the  curriculum 
development  process,  there  is  a  dearth  of  tools  for  systematic  data  gathering  and 
rational  analysis  that  might  inform  and  support  the  kind  of  collaborative  decision 
making  that  is  essential  for  effective  results.  Such  tools  are  especially  needed  in 
educational  institutions  that  subscribe  to  contemporary  theories  of  decentralized 
authority  and  management. 

Using  Maps  for  Curriculum  Development  


Fundamentally,  this  report  addresses  the  need  for  new  tools  by  investigating  an 
experimental  process  for  generating  curriculum  maps  from  concept  maps.  Since 
there  is  great  diversity  in  the  terminology  that  is  used  to  describe  conceptual  and 
curricular  structures,  we  begin  by  defining  three  types. 

Curriculum  maps  are  representations  of  the  structures  employed  by  schools 
and  other  educational  organizations  as  a  means  for  assigning  activities  and 
allocating  time  intended  to  facilitate  the  development  of  student  knowledge. 

Concept  maps  are  representations  of  the  structure  of  "public"  conceptual 
systems,  where  "public"  implies  shared  use  of  common  terminology  and  language 
including— but  not  restricted  to— forums  such  as  publications,  lectures  and  other 
presentations,  discussions,  and  other  formal  and  informal  communication. 

Cognitive  map*  are  representations  of  the  internal  mental  structures  of 
individuals  that  emphasize  the  meaning  that  individuals  attribute  to  concepts; 
thny  may  be  used  to  describe  expert  knowledge,  to  diagnose  or  assess  an 
iiulividunpH  understanding  of  a  knowledge  domain,  or  to  compare  and  contrast 
differences  in  the  conceptual  systems  of  individuals  or  groups. 

Curriculum  Maps 

Among  the  obstacles  to  systematic  curriculum  development  is  the  problem  of  sifting 
tli  rough  large  numbers  of  documents  relevant  to  curriculum  decisions:  state  and 
national  reports  and  curriculum  frameworks,  text  books,  journals,  existing  syllabi 
and  the  like.  Flowcharting  and  mapping  techniques  have  been  used  for  planning, 
implementing  and  evaluating  the  curriculum  but  computers  have  only  recently  been 
used  to  process  the  large  amounts  of  information  that  are  generated  during 
curriculum  development. 

Kenwick  English  (1980)  proposed  the  technique  of  "curriculum  mapping"  as  a 
means  of  quality  control  His  approach  emphasizes  assessment  of  the  extent  to  which 
learning  tasks  and/or  learning  time  conform  to  standards  delineated  in  a  curriculum 
guide,  and  attempts  to  determine  the  extent  to  which  what  is  actually  taught  is 
congruent  with  measures  of  achievement  A  step-by-itop  guide  for  curriculum 
mapping  can  be  found  in  Donald  F.  Weinstein's  Administrator'*  Guide  to  Curriculum 
Mapping  (  1986).  Curriculum  mapping  can  be  used  to  determine  potential  corrections 
and  improvements  in  school  programs.  Essentially,  English's  use  of  the  term 
"curriculum  mapping"  refers  to  a  comparison  (mapping)  of  an  idealized  curricular 
structure  with  an  attempted  implementation  and  we  think!  therefore,  that  a  better 
term  for  the  process  he  describes  would  be  curriculum  audit  mapping.  This  is 
consistent  with  English's  later  work  in  curriculum  auditing  (English,  1988).  We 
reserve  the  term  curriculum  map  for  each  of  the  two  structures  that  are  being 
compared:  the  desired/required  and  the  actual.  Our  report  is  concerned  with 
methods  for  developing  and  using  maps  of  the  desired/required  curriculum. 

Another  strategy  for  curriculum  development  and  management  can  be  found 
in  Warren  Hnthaway's  (1989)  use  of  PERT-like  networks  to  organize  the  curriculum 
and  to  manage  instructional  resources,  PEKT  (Program  Evaluation  and  Review 
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Technique)  charts  are  widely  used  in  business  and  industry  to  plan  and  monitor 
parallel  activities  in  support  of  common  goals  or  outcomes.  They  are  used  to  allocate 
and  monitor  resources,  to  estimate  completion  dates,  and  to  determine  ongoing 
progress.  Hathaway's  contribution  was  to  show  that  such  techniques  could  he 
applied  to  the  achievement  of  instructional  goals  as  well  aB  the  construction  of 
buildings  or  organization  of  factories. 

Concept  Maps 

In  contrast  to  the  systems  proposed  by  English  and  Hathaway,  which  are  directed 
towards  institutionalized  implementation  and  management,  are  approaches  that 
focus  on  the  conceptual  organization  of  curricula.  Typical  is  the  work  of  Novak  and 
Gowin  (1984)  who  have  demonstrated  how  concept  mapping  can  be  used  to  plan  a 
total  curriculum  as  well  as  specific  instructional  activities. 

Before  we  discuss  the  use  of  concept  maps  for  curriculum  development,  let  us 
briefly  examine  the  variety  of  concept  maps  and  mapping  strategies  proposed  in 
literature.  Concept  maps  demonstrate  structural  relationships  between  concepts. 
Several  types  of  structures/representations  such  as  spatial  maps  (Holley  and 
Dansereau,  1984),  nested  hierarchical  maps  (Miller,  1969;  Johnson,  1967),  graphic 
maps  (Anderson  and  Bower,  1973;  Quillian,  1968;  Rumelhart  et  al,  1972)  have  been 
described.  Preece  (1978)  has  reviewed  the  research  on  various  organizations  of 
concepts  in  these  maps.  Although  the  different  types  of  maps  reflect  different 
theories  of  semantic  structure,  Friendly  (1977)  suggested  that  the  models  were 
actually  quite  similar  in  their  use  of  propositions  and  semantic  distance  data. 
Nevertheless,  there  are  differences  in  the  methods  and  conventions  used  to  construct 
concept  maps.  We  will  briefly  examine  three  approaches  described  by  Holley  and 
Dansereau  ( 1984):  networking,  mapping,  and  schematizing. 

Holley  and  Dansereau  argue  that  networking  is  a  fundamental  tool  for 
acquiring  and  developing  individual  knowledge.  During  acquisition,  the  student 
identifies  important  concepts  or  ideas  in  the  material  and  represents  their  inter- 
relationships and  structure  in  the  form  of  a  network  map.  Students  are  taught  a  set 
of  named  links  that  can  be  used  to  code  relationships  between  ideas.  The  networking 
process  emphasizes  the  identification  and  representation  of  (1)  hierarchies,  (2)  chains 
of  logic,  (3)  clusters.  An  example  of  a  network  is  shown  in  Figure  1. 

Mapping  is  another  techniquo  used  to  represent  knowledge.  Several 
researchers  have  investigated  concept  mapping  strategies.  An  elementary  approach 
to  mapping  proposed  by  Hauf  ( 1971)  involves  placing  the  central  idea  of  a  passage 
near  the  middle  of  a  note  page  and  attaching  the  subsidiary  ideas  in  a  concentric 
fashion,  thus  producing  a  product  resembling  a  road  map.  According  to  Holley  and 
Dansereau,  mapping  requires  a  set  of  relational  conventions  or  symbols.  In  their 


(0  IW2,  Allen,  llulimun,  Kimii|h'IIii.  St  Shi  lit 


4 


ihtfv  \ 


scar  tissue 
is  weak 


take  step*  to 
minimize  scars 


►  incision  (sharp  cutting  instrument) 

•  abrasion  (scraping  or  rubbing) 

•  puncture-stab  (nail,  bullet) 

•  laceration  (blunt  instrument) 


I 


may  occur  in  iiny 
combination 


epithelial  cells  | 
grow  in  irom  | 
edges 


edecs 
C  scar  J 


small 
blood 
vessels 
disappear 
and  scar 
shrinks 

granulation  tissues 
(fibroblasts  and 

small  blood  vessels) 
grow  along  fibrin 
network  and 

gradually  absorb  it 

1 

|  tissue  continuity  | 

Figure  1.     Example  of  a  network  of  a  chapter  from  a  nursing  textbook  (from  Holley 
and  Dansereau,  1984). 

research,  they  provided  students  with  symbols  depicting  seven  fundamental 
relationships: 

1.  B  is  an  instance  of  A; 

2.  B  is  a  property  of  A; 

3.  A  is  similar  to  B; 

4.  A  is  greater  than  or  less  than  B; 

5.  A  occurs  before  B; 

6.  A  causes  B;  and 

7.  A  is  the  negation  of  B. 


An  example  of  a  concept  map  developed  by  Stewnrt,  Van  Kirk,  &  Howell 
Idled  in  Holley  and  Dansureau,  1084)  is  shown  in  Figure  2.  For  a  review  of 
relational  descriptor  Bee  Allen  and  Hoffman  (in  press). 
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Figure  2,     A  general  concept  map  for  ecology  (from  Stewart,  Van  Kirk,  &  Rowell, 
1979  in  Motley  and  Dansereau,  1984,  page  238). 

Novak  and  (lowin  { 1984)  draw  maps  by  linking  concepts  with  relations 
derived  from  the  text  rather  than  using  a  specific  set  of  relations.  These  relations  are 
known  as  propositions.  (Hauf,  and  Novak  and  Gowin  actually  use  the  verb  "concept 
mapping**  for  their  techniques.  However,  as  noted  earlier,  we  propose  to  use  "concept 
map '  as  a  noun  to  describe  a  much  more  inclusive  set  of  conceptual  representations.) 

Another  technique  for  knowledge  representation  described  by  Holley  and 
Dansereau  is  called  svhematizitm  involves  labeling  and  clustering  concepts  and 
depicting  relationships  between  concepts  by  lines  that  are  annotated  to  reflect  the 
seven  types  of  relation  ships.  The  general  process  for  schematizing  is  represented  in 
Figure  3. 


tuxl 


concepts 


labels 


relationships 


relationship  symbols 


schcnmli/alion 


Figure  3.  An  example  of  schemati/ation  (after  Holley  &  Dansereau.  1989). 

The  principal  difference  between  networking  and  mapping  appears  to  be  that 
mapping  emphasizes  local  organization  rather  than  abstraction  of  an  overall 
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framework  or  schema,  and  that  it  einployB  Bpatial  representations  of  relationships 
rathtir  than  labeled  relationships.  Schematization  is  similar  to  networking  and 
different  from  mapping  in  that  it  uses  annotated  lines  to  represent  relationships 
between  concepts  and  it  emphasizes  the  extraction  of  an  overall  framework  or 
mncrostructure.  It  is  different  from  networking  in  the  types  of  relations  depicted,  the 
method  of  annotation  used,  and  the  organizational  structure  of  the  resulting 
diagrams.  The  differences  between  these  mapping  strategies  are  superficial;  they  are 
more  similar  than  different.  We  have  used  "mapping"  in  the  title  of  this  report  in 
lieu  of  "networking0  in  order  to  avoid  confusion  with  the  use  of  electronic 
communication  networks  and  professional  networks. 

Networking,  mapping,  and  schematizing  have  been  applied  directly  or 
adapted  and  modified  to  initiate  various  cognitive  processes  and  achieve  various 
learning  outcomes.  Concept  maps  help  learners  to  extract  meaning  from  written  and 
oral  communication.  They  have  been  used  extensively  to  promote  meaningful 
learning.  Since  curriculum  devnlopment  is  a  mus^faceted  enterprise,  the 
advantages  of  concept  maps  for  curriculum  development  can  be  better  appreciated  if 
we  examine  the  characteristics  of  concept  maps  that  make  suitable  suitable  for  a 
variety  of  applications. 

Applications  of  Concept  Maps 

Concept  maps  have  been  used  for  understanding  text.  The  process  of  understanding 
text  can  be  thought  of  as  a  transformation  of  sequences  of  words,  sentences,  and 
paragraphs  into  a  coherent  conceptual  structure:  synthesized  knowledge.  Carl 
Frederiksen  (1975)  presents  a  network  model  of  logical  and  semantic  structures  from 
which  speakers  and  writers  generate  linguistic  messages  at  the  discourse  level.  Book 
webbing  is  an  extension  of  the  use  of  concept  maps  for  transforming  text  into  spatial 
representations.  Virginia  Nordstrom  and  Victoria  Clayton  (1988)  suggest  the  use  of 
a  form  of  concept  maps  to  integrate  multicultural  children's  literature  into  the 
curriculum.  Book  webbing  uses  children's  literature  to  develop  a  study  of  one  book; 
the  works  of  a  particular  author  or  illustrator;  a  genre  of  books,  such  as  poetry;  or  a 
topic,  such  an  travel  or  magic.  From  the  web's  core  based  on  a  specific  work  of 
children's  literature,  the  instructional  possibilities  of  the  book  spin  out  in  strands.  As 
the  plan  develops,  opportunities  emerge  to  individualize  instruction  to  meet  the 
needs  of  each  student's  learning  style  and  interest. 

Concept  maps  have  also  been  successfully  used  as  advance  organizers. 
Willerman  and  Harg  (1991)  conducted  an  experiment  to  determine  if  the  use  of 
concept  maps  es  advance  organizers  improved  the  science  achievement  of  eighth- 
grade  students.  They  found  that  a  concept  map  provided  teachers  with  a  meaningful 
and  practical  Htructured  approach  for  introducing  advance  organizers  in  the  class.  In 
thoir  study,  a  concept  map  developed  by  teachers  was  used  as  an  advance  organizer 
and  was  presented  to  students  as-a  visual  tool  for  organizing  content.  Willerman  and 
Harg  felt  that  the  students  were  helped  by  the  organization  and  visual  relationships 
of  the  advance  organizer  in  ways  that  are  more  effective  than  the  assistance  provided 
by  a  prose  passage  or  an  oral  explanation.  The  concept  map  assisted  the  teacher  in 
describing  the  relationships  between  important  ideas  and  learners'  knowledge, 

Concept  maps  have  been  used  as  a  guide  to  develop  hypermedia  applications. 
Authors  need  a  way  of  representing  knowledge  structures  so  that:  (a)  they  know  the 
relationships  between  concepts  and  therefore  do  not  make  unwarranted  leaps  or 
linkages  between  them;  and  (b)  end-users  can  see,  at  a  global  level,  the 
interrelationships  between  concepts.  If  end-users  can  see  the  overall  map  of 
knowledge;  then  they  can  make  informed  choices  about  the  order  and  sequence  of 
learning  (individualizing),  and  at  the  same  time  check  that  their  sequence  is  coherent 
with*  more  general  understandings.  Designers  often  resort  to  some  form  of  mapping 
to  analyze  the  content  and  sequence  of  instruction,  Students  in  the  Department  of 
Kduciitittnat  Technology  at  Han  I)ie||o  Htate  University  also  use  a  computer-based 
semantic  net  work  tool  to  map  out  concepts  and  relations  in  a  particular  knowledge 
domain.  Hlgurw  4  is  an  example  uf  a  semantic  network  used  to  analyze  content  for  a 
b/ochemistry  laboratory  course. 
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Figure  4.     An  example  of  a  semantic  network  used  to  analyze  content  for  a 

multimedia  course  on  biochemistry.  The  display  is  from  a  SemNet 
file.  Clicking  on  the  P,  Tt  or  S  tabs  activates  supplementary  fields  for 
pictures,  text,  or  synonyms  (from  Allen  &  Hoffman,  in  press). 


Cognitive  Maps 

Cognitive  maps  may  be  described  as  representations  of  an  individuals  knowledge. 
Learning  occurs  when  students  link  new  concepts  into  more  comprehensive 
structures  and  patterns,  Bach  learner's  associative  patterns  are  unique  and  it  may 
be  useful  to  examine  them  in  terms  of  the  concepts  the  learners  emphasize,  the 
relations  that  link  the  concepts,  and  the  strengths  of  these  links.  One  study  in  this 
area  was  conducted  by  Kathleen  Fisher  (1989)  who  analyzed  biology  nets  built  by  her 
students  and  obtained  insights  into  the  students1  cognitive  abilities  that  would  have 
been  difficult  with  conventional  assessment  methodology.  Comprehensive  essay 
questions  do  yield  some  information  about  the  students'  understanding  of  the  subject 
matter;  however,  analysis  is  problematical. 

Cognitive  maps  are  also  an  excellent  tool  for  representing  the 
teacher's/expert's  knowledge.  According  to  Shulman  (1986),  teachers  need  both 
content  knowledge  and  pedagogical  expertise.  The  types  of  content  knowledge  they 
require  can  be  broken  into  three  categories: 

•  subject-matter  content  knowledge; 

•  subject-matter  pedagogical  knowledge; 

•  curricular  knowledge. 

All  three  types  of  knowledge  can  bo  analyzed  and  synthesized  using  concept 
maps.  In  developing  subject  matter  content,  teachers  must  go  beyond  the  facts  and 
concepts  in  a  domain  to  consider  patterns  that  reflect  the  substantive  and  syntactic 
structures  of  knowledge.  These  patterns  help  the  teacher  to  convey  the  concepts  in 
the  domain.  Perhaps  even  more  important,  they  help  the  teacher  to  explain  why  a 
particular  concept  is  worth  knowing  and  how  it  relates  to  theoretical  and  practical 
issues  both  within  the  discipline  and  without.  In  the  category  of  subject-matter 
pedagogical  knowledge,  Shulman  includes  useful  forms  of  representation  such  as 
illustrations,  analogies,  examples,  demonstrations,  etc.  These  can  be  mapped 
prescriptively  using  relational  descriptors  that  specify  causal  relationships  between 
required  learning  outcomes  and  recommended  strategies.  The  third  category, 
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curricular  knowledge,  is  knowledge  of  the  full  range  of  programs,  materials,  and 
facilities  available  for  teaching  different  concepts  at  various  levels. 

Shulman's  categorization  of  a  teacher's  knowledge  lends  itself  to  an 
explanation  of  our  method.  The  characteristics  of  concept  maps  that  have  been 
exploited  for  other  applications  can  be  brought  to  bear  in  rationalizing  the  curriculum 
design  process.  Concept  maps  can  be  used  to  show  concepts  and  relations  explicitly 
for  subject-matter  content  analysis.  Pedagogical  strategies  can  be  incorporated  into 
the  concept  maps.  Time  and  task  allocations  can  be  made.  On  the  other  hand,  the 
curriculum  designers  and  experts  can  incorporate  their  cognitive  structures  also  into 
these  maps.  Thus  curriculum  developers  can  create  what  we  will  call  integrated 
curriculum  knowledge  maps  to  first  input  the  knowledge  base,  then  reorganize  it  to 
suit  the  instructional  objectives  and  finally  extract  a  curriculum  that  can  be  justified. 
These  integrated  maps  can  be  successfully  used  by  curriculum  developers  to  organize, 
analyze,  and  synthesize  the  information. 

Integrated  Curriculum  Knowledge  Maps 

In  previous  sections,  we  have  discussed  three  approaches  to  mapping  knowledge: 
curricular  maps  are  concerned  with  knowledge  as  it  is  represented  by  institutions  for 
the  convenience  of  educational  delivery  systems;  concept  maps  are  concerned  with 
knowledge  as  it  is  publicly  represented  by  a  community  that  specializes  in  a 
particular  knowledge  domain;  and  cognitive  maps  are  concerned  with  knowledge  as  it 
is  represented  in  the  minds  of  individuals.  There  are,  of  course,  overlaps.  For 
example,  the  creation  and  interpretation  of  concept  maps  is  obviously  influenced  by 
the  unique  cognitive  structures  of  individuals  who  author  the  maps.  On  the  other 
hand,  the  assessment  or  diagnoses  of  individual  student  understanding  through 
cognitive  maps  can  be  influenced  by  the  concept  maps  used  to  represent  the 
"knowledge  domain.11 

This  report  is  concerned  with  another  type  of  overlap,  or  rather  derivation: 
the  development  of  curricular  maps  from  concept  maps.  Although  the  use  of  concept 
maps  for  curriculum  development  is  not  new,  the  representation  of  large  amounts  of 
information  in  the  form  of  a  map  that  is  suited  for  navigation,  psrusal,  and 
processing  is  a  new  type  of  software  engineering  problem.  Knowledge  representation 
is  one  the  of  the  central  issues  in  knowledge  engineering,  a  rapidly  advancing  field 
that  offers  many  techniques  (including  concept  mapping)  with  potential  relevance  to 
curriculum  development  (for  a  review,  see  Tuthill,  1990).  However,  to  be  practical  for 
use  in  curriculum  development,  we  believe  such  techniques  must  be  brought  to  the 
level  of  "desktop  computing11  i.e.,  usable  by  professional  educators,  administrators, 
and  other  leaders  whose  primary  expertise  is  not  computer  programming  or 
information  processing. 

Tools  for  computerised  curriculum  development  must  effectively  support 
elicitation  and  organization  of  relevant  knowledge  from  a  variety  of  sources  including 
reports,  books,  subject-matter  experts,  teachers  and  administrators.  Such  tools  must 
also  support  analysis  of  relationships  between  curricular  elements  such  as 
competencies  and  learning  activities  and  the  synthesis  of  structures  familiar  to 
educational  institutions:  courses  prerequisites,  scheduled  class  sessions,  etc,  They 
need  to  be  supportive  of  collaborative  work,  open  to  modification  and  they  must  have 
a  "learning  curve'1  that  does  not  impose  an  unnecessary  burden  on  development 
specialists. 

Such  tools  have  only  recently  become  commercially  available  in  the  form  of 
computer-based  graphical  mapping  tools  which  represent  complex  relational  data  us 
n-dirnensional  networks.  One  example,  used  extensively  in  this  study,  is  HwiNet 
developed  by  the  SemNet  Research  Group  (199 1).  SemNet  serves  as  u  I'ritmowork  for 
modeling  human  semantic  memory— described  by  Tulving  ( 19721  as  "a  munt.nl 
thesaurus,  organized  knowledge  a  person  possesses  nlmut  Words  intil  other  Verba! 
symbols,  their  meaning  and  referents,  about  relations  among  them,  mnl  lllmiit.  ruins, 
formulas,  and  algorithms  for  the  manipulation  of  these  symlmls,  coineplH,  and 
relations."  Although  cognitive  psychologists  have  proposed  several  other  type*  of 
memory,  semantic  memory  is  widely  seen  as  crucial  to  the  m't|iil*|ttmi,  ptnuiMlng, 
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and  retrieval  of  verbal  knowledge  and  therefore  central  to  learning,  especially  in 
school-like  settings. 

Semantic  memory  has  been  known  to  be  associative  since  the  time  of 
Aristotle.  However,  it  is  only  in  the  last  two  decades  that  associative  memory  has 
taken  a  firm  hold  with  those  interested  in  modeling  human  memory  or  providing 
working  memories  for  "intelligent"  computer  programs.  Semantic  networks  typically 
organize  knowledge  in  terms  of  two  types  of  elements:  concepts  and  relations. 
Concepts  are  not  easily  defined  but  include  a  wide  variety  of  entities  such  as  ideas, 
objects,  notions,  beliefs,  events,  features,  properties  and  states.  For  the  purposes  of 
this  project,  a  concept  represents  anything  that  can  be  assigned  an  identity.  New 
concepts  are  defined  by  relating  them  to  pre-existing  concepts. 

ScmNet  uses  object-oriented  programming  and  the  Macintosh  graphical 
interface  to  display  labeled  nodes  (referred  to  as  "concepts")  and  labeled  arrow-links 
(called  "relations").  Node-link-node  sequences  are  called  "instances. "  It  can  be  used 
to  represent  a  wide  variety  of  domains  to  the  extent  that  they  can  be  represented  as 
semantic  networks  or  propositional  networks.  In  typical  browsing  maneuvers,  users 
nee  a  central  node  connected  to  its  immediate  "satellites"  (see  Figure  5).  Clicking  on 
n  satellite  node  "moves"  the  frame  so  that  the  satellite  assumes  the  center  position 
thus  revealing  its  satellites.  In  the  process,  the  visible  label  for  asymmetrical  links 
will  change.  For  example  "formative  evaluation — ISA— >  evaluation"  becomes 
"evaluation — is  a  kind  of— >  formative  evaluation*' 

Moving  through  a  sequence  of  alternating  nodes  and  links  generates  a  "path" 
which  can  be  circuitous,  that  is  a  path  that  entails  a  single  node  several  times.  (This 
results  when  the  user  visits  the  same  concept  repeatedly.)  Users  can  also  move 
quickly  and  intuitively  through  large  nets  by  selecting  items  in  indexes,  or  by 
invoking  utility  commands  such  as  "jump"  and  "replay  path." 

Easily-used  commands  afford  several  techniques  for  creating  and  modifying 
connections;  experienced  users  can  construct  nets  at  rates  exceeding  several 
instances  per  minute  if  they  are  clear  about  how  to  organize  their  ideas.  Users  can 
assign  graphics  and  short  text  passages  to  nodes  and  links  (and  in  future  versions 
will  be  able  to  assign  video  and  audio  segments  to  nodes).  They  can  also  export 
graphics  and  ASCII  text  representing  indexes  or  net  contents  and  subsets  thereof. 

The  program  provides  a  variety  of  metrics  about  the  size  and  complexity  of  a 
net  and  uses  these  to  assemble  various  types  of  summaries  and  overviews.  Almost 
anyone  who  browses  through  a  SetnNet  expresses  interest  in  viewing  "the  whole  net." 
This  has  proven  problematical  since  SemNets  are  n-dimensional  structures  that 
cannot  easily  be  visualized  in  two  dimensions.  But  the  problem  has  been  addressed 
by  special  displays. 
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Figure  5.     Concept  map  from  a  course  on  instructional  design  (from  Allen  & 


Fur  uxumple,  measures  of  the  "embeddedness"  of  concepts  are  used  to 
generate  n  'knowledge  core"  in  which  the  most  embedded  nodes  are  laid  out  in  rows 
and  connected  by  crisscrossed  lines  (see  Figure  6).  Other  utilities  let  users  see  the 
.shortest  path  between  two  nodes,  or  traverse  the  net  viewing  only  node-and-satellite 
frames  that  meet  user-established  criteria. 

Because  SemNet  uses  a  flexible,  multi-dimeasiona!  graphic  mapping  strategy, 
it  can  be  used  for  integrated  representations  of  curricular  maps,  concept  maps,  and 
cognitive  maps,  and  it  can  be  used  to  derive  structures  in  one  map  from  structures  in 
another.  The  experiments  described  here  focus  on  deriving  curriculum  maps  from 
concept  maps.  It  is  also  possible  to  use  SemNet  to  explore  relationships  between 
concept  maps  and  curriculum  maps  one  hand  and  individual  cognitive  maps  on  the 
other,  hut  this  in  a  subject  for  another  report. 

Development  of  an  Integrated  Curriculum  Knowledge  Map 

To  explore  the  possibilities  of  integrated  curriculum  mapping,  a  team  of  students 
drawn  primarily  from  the  SDSU-Claremont  Graduate  School  Joint  Doctoral  Program 
in  Multi-Cultural  Education  assembled  under  the  direction  of  Dr.  Brock  Allen  at  San 
Diego  Statu  University.  The  framework  for  their  efforts  was  provided  by  Dr.  Thomas 
Sticht,  President  of  Applied  Behavioral  and  Cognitive  Sciences.  Dr.  Sticht  is 
developing  a  design  for  a  model  degree  in  Workforce  Education  and  Life-long 
Learning. 

The  proposed  WELL  degree  program  will  be  aimed  at  producing  educators 
capable  of  delivering  basic  knowledge  and  skills  required  for  the  workplace  of  the 
future.  WELL  Specialists  are  to  organize  and  implement  programs  for  the 
educationally  underserved.  This  population  includes  high  school  dropouts,  those 
with  inadequate  literacy  skills  for  acquiring  or  maintaining  a  job,  or  those  with  jobs 
who  arc  immobilized  by  the  lack  of  such  skills. 
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processes  by  identifying  patterns  in  tho  raw  data,  unencumbered  by  categorization 
imposed  by  data  sources. 

An  experimental  process  was  adopted  to  accomplish  the  team  s  goals.  This 
process  was  modified  and  refined  during  the  course  of  the  project,  and  from  the 
team  s  experience  we  have  extracted  an  idealized  six-step  model  (Figure  7). 

Process  Description 

The  project  began  with  a  review  of  the  social  and  political  impetus  for  the 
model  program,  along  with  pertinent  research.  Three  sub-committees  were 
established  to  catalog  descriptions  of  the  knowledge  WELL  Specialists  would  need, 
Incusing  in  three  areas:  cognitive  foundations  of  education,  educational  arts  and 
technologies,  and  human  and  cultural  resources.  Each  sub-committee  used  SemNet 
In  entei  knowledge  descriptors  into  a  domain-specific  subnet. 

The  three  subnets  were  then  combined  so  the  entire  team  could  collaborate  to 
edit  t  In*  main  net  and  explore  strategies  for  generating  useful  information  from  the 
net   Divergent  terminology  iti  both  knowledge  descriptions  and  the  relations  that  are 
used  to  link  t  hem  were  reconciled  as  the  subnets  were  combined  into  the  main  net. 
Merging  was  accomplished  by  manually  reentering  the  subnets  into  a  main  net. 
More  efficient  merging  capabilities  have  since  been  added  to  the  SemNet  program. 

A  Macintosh  computer  lab  with  an  LCD  overhead  projector  enabled  team 
members  to  work  together  and  to  share  nets  and  net-building  techniques.  Timbuktu 
(Killen  et  al,  1000)  software  allowed  multiple  users  to  control  the  activity  of  a  single 
host  computer.  Outside  the  lab,  traditional  methods  of  file  sharing,  namely,  floppy 
disk  swapping  and  exchanging  files  via  modem,  were  utilized. 

1 .  Gather  Relevant  Descriptions  of  Knowledge 

In  the  first  step,  subcommittees  UHod  SemNet  to  catalog  knowledge  descriptors 
gleaned  from  a  variety  of  nourceK  including  reports,  textbooks,  subject  matter  experts, 
and  existing  course  syllabi.  For  the  purposes  of  this  study  we  define  a  knowledge 
descriptor  as  a  label  for  a  concg|tf  or  skill  (or  sets  thereof)  which  may  be  of  value  to 
WELL  Specialists  in  their  professional  practice. 

2.  Construct  Hierarchical  Links  between  Knowledge 
Descriptors 

In  the  second  step,  hierarchical  relations  were  used  to  represent  the  topical 
structures  of  the  various  source  documents.  This  permitted  efficient  input  of 
typically  hierarchical  sources  such  as  reports,  textbooks  and  course  syllabi.  It  also 
enabled  later  extraction  of  the  hierarchical  structures  in  outline  form  using  a  SemNet 
hierarchical  outline  utility. 

Hierarchical  structures  were  entered  prior  to  heterarchical  ones  when 
representing  a  given  document.  Consensus  for  this  strategy  emerged  from  divergent 
approaches  taken  by  different  subcommittees.  One  committee  entered  hierarchical 
and  heterarchical  relations  concurrently.  An  instance  entered  with  a  heterarchical 
relation  was  not  necessarily  entered  again  with  a  hierarchical  relation.  This 
precluded  later  extract  ion  of  the  entire  document  outline  using  hierarchical  relations. 

A  Hecond  committee  its  work  by  cataloging  knowledge  descriptors  from  course 
syllabi,  using  only  hierarchical  relations  at  first.  Later  they  used  heterarchical 
relations  to  "cross-link"  the  knowledge  the  knowledge  descriptors.  The  hierarchical 
structure  reflected  in  the  original  source  documents  thereby  remained  intact  and 
could  be  extracted  even  after  extensive  heterarchical  cross-linking  (Figure  8). 
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Kigurt'  7.      An  itlnnl i/.ed  six-step  model  for  using  n  computer-based  semantic 
tifl  work  program  to  construct  on  integrated  curriculum  maps. 


3.    Construct  Heterarchical  Links  between  Knowledge 
Descriptors 

The  third  step  employed  heternrchical  relations,  such  as  causal  and  procedural 
relationships,  to  cross-link  the  knowledge  descriptors  within  and  among  documents 
(I'Tgurr  0)  This  step  was  carried  out  initially  in  the  subnets  and  then  again  after 
the  nets  were  merged  into  the  main  net. 

The  strategy  for  interconnecting  nodes  is  one  way  to  counteract  the  effect  of 
lop-down  bias  stemming  from  prior  categorization  imposed  by  the  original  authors  of 
the  descriptions  of  knowledge.  The  various  syllabi,  documents,  and  material  from 
subject  matter  experts  all  came  fanned  in  their  own  categorical  structures.  The 
interconnect  process  was  designed  so  that,  hopefully,  natural  centers  of  gravity  would 
emerge  from  the  mass  of  knowledge  descriptors  in  the  main  net  itself,  rather  than 
from  these  prior  categorical  structures  To  accomplish  thin,  the  teams  first  attended 
to  those  concepts  that  weie  hast  embedded  or  linked  to  other  concepts  in  the  net  It 
was  assumed  that  later,  once  these  simple  units  had  been  linked  into  compound 
structures,  new  patterns  might  become  evident. 

SetoiNet  screens  list  concepts  by  the  number  of  branches,  by  emheddedness, 
and  in  alphabetical  order  Studying  the  list  of  concepts  arranged  in  order  of  the 
number  of  branches  or  links  to  other  concepts  can  help  the  curriculum  developer  gel 
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Figure  8.     This  first  port  ion  of  an  outline  generated  by  SemNet  beginning  on  the 
"Educational  Technology"  and  based  on  the  relation  "has  topic." 


node 


Although  many  of  these  nodes  have  been  extensively  cross-linked  to 
other  parts  of  the  net,  the  hierarchical  structures  of  the  original  course 
syllabi  have  been  extracted  by  isolating  concepts  linked  by  the  "has 
topic"  relationship.  Limitations  on  the  size  of  labels  for  concepts  in 
SemNet  necessitated  abbreviation  of  knowledge  descriptors 
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Figure  Bi     An  example  of  a  moderately-connected  or  "compound"  concept, 

"Recognition"  is  linked  in  a  variety  of  ways  to  other  nodes  within  the 
curriculum  net. 

a  sense  of  what  might  be  the  more  important  concepts  (Figure  10).  If  a  concept  is 
important,  or  central,  tn  a  domain,  it  is  presumed  to  have  many  direct  connections  to 
other  concepts. 

Listing  concepts  in  order  of  emheddeduess  offers  further  evidence  of 
ceiitrulitv  or  importance  For  example,  if  a  concept  has  three  related  concepts,  but 
each  of  those  three  concepts  average,  say,  three  related  concepts  each,  then  the 
original  ennrept  is  said  to  have  an  embeddedness  of  3  x  3  +  3  =  12. 
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Lists  cf  concepts  in  creation  nnd  alphabetical  orders  were  used  more  as  net 
editing  tools  than  as  analytical  tools.  Creation  order  enabled  later  identification  of 
separate  documents  represented  within  the  net.  Team  members  inserted  marker 
nodes  between  documents  to  help  in  later  distinguishing  the  sources  of  various 
competencies.  Alphabetical  order  helped  search  for  specific  compel eurles  during  flu* 
cross-linking  process. 


Concept*  by  Number  of  ht»t«m 

46  WELL  Specialist 

31  SDSU  Leorning  Activity 

27  literacy 

26  Literacy  Delivery  ByRtem 

21  problem  solving 

19  multicultural  paradigm 

19  administratinn 

18  cognition 

17  action  research 

16  thinking  and  reasoning 

16  te  423  instructional  tnpirs 

16  edtec  700  cogn.  Inst.  Ujpira 

15  perception 

15  restructuring 


1  learning  environment 

1  content  expertise 

1  bosic  education  (non-Rclino! 

1  Labor's  Perception 

1  articulate  educative  needs 

1  inferential  study 

1  bottom-up  processing 

1  equipment 

1  success 

1  tools  for  representation 

1  speaking 


IttUH  Hums 

11(1 
HI 
I7H 
I  .'III 

711 
I  OH 
1V*8 
146 

i  mi 

44 

r?. 
h:i 

ri 


12 

\\ 
H 

in 
ll 

;i 

i 

12 
:i 

12 


Figure  10.    The  beginning  and  end  portions  of  the  list  ni  concepts  in  order  of  the 
number  of  branches  or  instances  The  heading  indicates  that  the  net 
consists  of  1248  items  or  nodes  The  nuinher  In  the  leH  of  each  line  of 
the  list  is  the  number  of  direct  links  or  branches  between  Ihe  listed 
node  and  other  nodes  in  tin1  net,  The  number  to  the  i  ight  indicates 
"emheddedness/1  which  adds  to  the  nunibei  of  direct  blanches  the 
number  of  the  branches'  branches. 


Cross-linking  of  the  suhnetB  was  carried  out  within  mihcommittt'OH  using  a 
single  computer  and  the  print-outs  described  above.  AHor  merging  ihe  subnets,  a 
somewhat  different  strategy  was  employed  to  cross-link  within  the  main  net.  This 
took  place  in  the  computer  lah,  where  each  subcommittee  had  access  to  two  or  three 
computers.  One  computer  in  each  subcommittee  provided  access  to  the  master  net 
through  Timbuktu  (Killen  el  al,  19510).  Since  only  one  computer  could  actually 
control  the  master  net  at  any  given  tune,  a  token  was  rotated  among  the 
subcommittees  so  that  all  would  know  where  on  line  editing  activity  was  taking 
place. 

The  token  was  passed  between  subcommittees  about  every  ton  minutes  or  at 
a  natural  break  in  activity.  During  the  twenty  minutes  or  so  when  the  token  wan 
elsewhere,  each  subcommittee  identified  those  nodes  in  their  net  that  could  be  linked 
to  other  concepts  in  the  master  net   To  aid  in  identifying  potential  links,  a  second 
computer  in  each  subcommittee  contained  an  off-line  copy  of  the  master  net  as  a 
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reference  and  a  third  computer  contained  a  reference  copy  of  the  subcommittee's 
subnet.  The  subcommittees  also  used  various  printed  lists  of  concepts  dorived  by 
using  using  SemNet's  "traverse"  utility. 

Typically,  while  a  subcommittee  was  "ofWine"  its  members  worked  with  a 
list  of  concepts  arranged  in  order  of  the  number  of  branches.  They  worked  from  the 
bottom  of  the  list,  beginning  with  the  least-connected  concepts,  Taking  each  in  turn, 
they  scanned  the  alphabetized  list  of  concepts  from  the  master  net  to  identify  several 
possible  links.  For  example,  the  concept  "school  restructuring"  seemed  to  be  related 
to  the  concepts  "educational  reforn  "  and  "organizational  change"  thai  had  been 
entered  by  other  subcommittees. 

In  order  to  chuck  the  tnr:;ntng  of  all  three  concepts,  the  subcommittee 
"jumped"  to  each  in  turn  on  the  off-line  copy  of  the  master  net.  They  discovered  that, 
indeed,  the  concept  "restructuring"  appeared  to  be  related  to  the  more  inclusive 
concept  "educational  reform,"  based  on  the  contextual  links  of  the  latter.  A  note  was 
made  to  link  them.  However,  the  concept  "organizational  change,"  seemed  to  have  a 
very  specific  context  having  to  do  with  workplace  issues,  and  a  link  was  thought 
inappropriate  Ah  oilier  concepts  were  found  to  have  important  or  interesting 
connections  in  the  master  net  .  the  subcommittee  kept  notes  until  the  token  arrived. 
They  then  took  control  of  the  on-line  master  net  and  entered  their  changes.  Concept 
by  concept,  the  ideas  became  gradually  more  interrelated  and  embedded  in  the 
master  net.  On  occasion  there  was  doubt  concerning  the  existence  or  meaning  of  a 
concept  and  the  subcommittees  would  instigate  oral  communications  in  an  effort  to 
clarify  or  reBnlve  differences 

After  linking  these  simple  concepts  from  the  three  original  subnets  into 
meaningful  clusters  or  "compound  concepts,"  the  subcommittees  turned  to  making 
links  between  the  compound  concepts.  (Many  of  these  had  been  formed  earlier 
during  development  of  the  separate  subnets.)  Linking  compound  concepts  produced 
highly  embedded  nodes  that  seemed  to  function  as  centers  of  "conceptual  gravity" 
(Figure  11).  The  process  of  linking  compound  level  nodes  was  conducted  in  the 
same  fashion  as  that  for  linking  simple  nodes. 

It  should  be  noted  that  the  student  participants  in  this  process  were  not 
subject  matter  experts  in  adult  education  or  educational  psychology  and  therefore  the 
representations  in  Figures  11  and  12  are  perhaps  a  bit  naive.  Our  concern  in  this 
study  was  primarily  with  the  process  of  curriculum  development  rather  than  content 
per  se. 

4.   Eetabliflh  "Course  Development  Nodes"  aud  Attach  Highly 
Embedded  Noden 

In  this  step,  teams  developed  structures  for  potential  courses  in  the  masters 
degree  program.  Of  some  1/200  coticcptti,  the  master  net  yielded  48  with  ten  or  more 
direct  links  to  other  concepts.  Kxamination  of  the  48  nodes  revealed  that  many  could 
be  thought  of  as  central  concepts  in  a  master's  degree  program.  Although  time  was 
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Figure  II     An  example  of  a  highly  embedded  node.  "Perception"  is  connected  to  a 
number  of  other  nodes  with  a  variety  of  relations.  Those  nodes  listed  in 
hold  type  have  additional  connections  to  other  nodes,  which  may  be 
displayed  by  double-clicking  on  them.  The  number  of  branches  from 
"perception  would  identify  it  as  a  potentially  important  concept  for 
Hubiumption  under  a  "course"  node. 

limited,  the  leant  experimented  with  creating  a  three-unit  course  and  in  less  than  an 
hour  of  discussion,  they  identified  nearly  a  dozen  thematic  concepts  that  could  be 
Buhsumed  under  a  node  they  called  "foundations  of  psychological,  development," 
(Figure  12)  including  such  themes  as  "perception"  and  "cognitive  development." 
This  subsumption  node  was  suggested  as  a  tentative  basis  for  a  course  or  aerie*  of 
courses  within  the  proposed  degree  program.  This  node  was  developed  in  a  period  of 
three  hours  by  students  without  extensive  expertise  in  psychology.  Therefore  it  ik 
illustrative  of  the  course  development  process  and  cannot  be  considered  as  a 
recommendation  for  actual  course  content.  However,  it  could  serve  as  a  rough 
framework  for  further  discussion  with  content  experts. 

5.  Attach  Activitiu*  and  Allocate*  Resources 

The  next  step  was  to  link  various  course  themes  to  a  battery  of  suggested  learning 
activities.  For  example,  a  theme  dealing  with  visual  phenomenon  might  rely  on 
textbooks,  demonstrations,  computer  aided  inst  ruction,  and  laboratory  activities.  By 
identifying  trends  in  learning  activities  that  might  be  appropriate  to  the  mix  of 
subject  matter  projected  for  the  program,  allocation  of  facilities  and  resources  would 
Ih>  facilitated. 
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Figure*  12,    Psychological  foundations  for  teaching  "potential  punk  performer*"  was 
established  as  a  subsumption  node  and  then  linked  to  a  number  of 
themes,  including  "perception"  (see  Figure  11). 

H.   Helect  and/or  Nequenee  Course  Development  Node* 

A  number  of  proposals  for  using  the  main  net  were  generated,  in  addition  to  course 
ami  syllabus  compilation  and  resource  allocation,  both  mentioned  above,  (he  net 
could  be  used  by  counselors  ami  Bturiettts  to  develop  an  individualized  com  He  of 
hI udy  II  could  also  lm  used  to  help  prospective  students  make  course  selection 
deriainus 

Ualng  the  Integrated  Curriculum  Knowledge  Map:  A  Bcenarto 

Wilb  Ibe  completion  of  !b»  sis-step  process,  the  main  net  could  be  considered  by  out 
definition  to  be  an  integrated  curriculum  knowledge  map  since  it  incorporated 
structures  that  represented  concept  mapping  (clusters  of  knowledge  descriptors)  as 
well  as  curriculum  mapping  fa  course  and  clusters  of  course  activities).  To  further 
explore*  the  implications  ofthfi  integrated  niap  as  a  framework  for  the  proposed 
wkUi  program,  each  of  the  team  members  worked  with  us  to  construct  a  scenario 
that  addressed  some  practical  problem  in  course  development  or  individualized 
learning.  Included  here,  in  edited  form,  is  one  of  these  scenarios;  it  should  be 
considered  tentative  and  exploratory. 

Rebbeca  Kvederis,  who  directs  career  counseling  services  at  Palomar 
Community  College  in  California,  describes  how  the  integrated  curriculum 
knowledge  map  could  be  used  to  counsel  prospective  WELL  Specialists  on  the 
suitability  of  the  master's  degree  program  for  their  needs,  capabilities,  and  interests. 
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She  also  examines  the  possibilities  that  the  map  oilers  lot  guiding  individualized 
study  plans. 

Using  Integrated  Curriculum  KnuwUdyc  Map*  in  Affftiluittli'  I'lHitnwIlitK 
Developed  with  Rebecca  Kvederis 

The  essence  of  all  counseling  relationships,  whidhei  limised  on  personal,  ncodtmitt, 
or  career  issues,  is  to  help  the  client "cope  effectively  wllh  nn  iin|icirl nut  problem  or 
concern,  to  develop  plans  and  moke  important  decisions  to  bring  nhout  a  desired 
future,  to  acquire  information  about  self  ami  relevant  aspect*  or  the  environment, 
and  to  explore  and  consider  options  available"  (Elsenberg  &  l)i  liuiey,  11177,  p  :i) 
College  academic  counseling  is  a  process  in  which  an  academic  counselor  assists  a 
student,  or  prospective  student,  in  planning  a  program  of  similes.  Counseling 
involves  consideration  of  the  current  skills  of  the  student  (through  |iurtroltn  review, 
asBCssment  test  scores,  etc.),  the  Interests  or  the  student  in  terms  of  career  and  jnnjor 
goals,  and  the  selection  of  courses  from  among  those  available  to  meet  degree 
requirements. 

Academic  counseling  is  rarely  a  one-time  event  t  'allege  students  request 
counseling  as  a  means  of  gaining  preliminary  information  about  the  options  available 
to  them  prior  to  engaging  in  educational  planning.  Thus,  the  initial  stage  in  this 
interaction  is  exploration  and  information  gathering,  leading  eventually  to  tin* 
development  of  an  educational  plan.  A  plan  nlay  be  retained  or  it  may  be  rejected, 
often  several  plans  are  "tried  on  for  size"  before  one  is  committed  lo 

Once  an  educational  plan  is  written  by  the  counselor  and  accepted  by  the 
student,  it  is  implemented.  Implementation  of  Hip  plan  (embarking  on  the  course  of 
study)  leads  to  new  experiences  and  interests  that  may  result  in  fin  titer  revision  of 
the  plan. 

If  the  educational  plan  is  a  dynamic  process,  so  is  the  counseling  interaction 
that  produces  it  The  counselor  describes  the  requirements  and  the  various  optiona 
lo  the  student  and  gauges  the  individuals  inteiest  baaed  on  verbal  (and  non  verbal) 
responses.  Certain  patterns  and  tendencies  are  noted  in  terms  of  the  students 
interests.  These  interest  patterns  influence  the  suggestions  made  by  the  counselor  to 
the  student  regarding  course  selection 

The  process  of  academic  counseling  ami  educational  planning  in  limited  by 
the  available  counseling  tools,  all  of  which  are  sequential  and  linear  ■  alnlogs, 
evaluation  forms,  and  checklists  of  requirements  These  eiuomage  students  to  be 
passive  in  the  counseling  interaction  since  they  establish  an  exportation  thai  an 
authoritative  list  of  requirements   ill  be  "given"  to  the  student  by  the  counselor,  aiiIi 
little  input  from  the  recipient.  A  tool  that  moves  away  liom  the  hadilinual, 
sequential  presentation  of  information  ami  innupotiiles  the  lealines  of  hypermedia 
would  not  only  offer  a  mote  accurate  lepteseutation  ot  reality  but  would  Im  dilate 
this  dynamic  process.  SmhNvI  is  such  a  fool 

SvmNvt  facilitates  representation  of  various  edu<  alional  domains  Nudes 
represents  clusters  of  skill  or  competencies  "Nuhsumptinn  nudes"  ran  be  uned  In 
collect  these  nodes  into  traditional  course  structures  and  to  assign  learning  activities 
to  such  structures  SvmNtt  affords  flexible  representations  cut  rimlat  options  \\\ 
"traversing''  the  net,  the  counselor  and  student  can  see  and  then  select  the  courses  or 
learning  activities  that  would  facilitate  development  of  I  be  needed  competencies. 

Using  SYm/Ve/  in  counseling  has  other  potential  benefits  A  student  who  Ik 
more  actively  involved  in  the  educational  planning  process,  who  is  taught  to  use  the 
net  to  explore  other  options  on  his/her  own,  is  more  likely  to  see  him/her  pelf  as 
responsible  for  and  capable  of  planning  an  educational  program.  Also,  the  net  may 
function  as  a  sort  of  job  aid  lor  the  counselor,  cueing  them  to  mention  specific  details 
related  to  the  node  under  consideration. 

The  following  is  a  hypothetical  case  study  constructed  to  show  how  SemNet 
would  be  used  in  an  academic  counseling  interaction  with  the  goal  of  designing  a 
course  of  study  for  a  graduate  student  entering  the  proposed  Workf  orce,  Kducutloti, 
and  Life-Long  Learning  Specialist  masters  degree  program. 
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The  WELL  academic  counselor  is  meeting  with  Jos6  Luna,  a  third-grade 
teacher  (or  the  past  flfluen  years  in  the  Ban  Marcos  School  District.  Mr.  Luna  desires 
it  new  professional  challenge  and  wants  to  work  with  adults.  He  believes  that  his 
bilingual  hMIIs  will  make  him  especially  valuable  as  an  instructor  in  the  district's 
adult  educat  ion  program  His  goal  is  to  become  a  program  supervisor  and  he  is 
especially  Interested  in  a  position  that  would  allow  him  to  continue  a  small  research 
project  he  began  several  years  ago  which  looks  at  parental  attitudes  and  the 
connection  Lo  children'**  learning  successes  among  the  Hispanic  population.  His 
research  ideas  stem  from  the  theoretical  concepts  he  learned  as  an  undergraduate 
psychology  major  and  Mexican-American  Studies  minor. 

The  initial  appointment  is  scheduled  to  begin  the  dialogue  between  counselor 
and  client  and  to  discuss  available  opportunities.  The  counselor  and  client  would 
view  the  WELL  Specialist  net  together,  using  the  "traverse"  function— beginning 
with  some  or  the  basic  knowledge  domains  and  moving  from  the  general  to  the  more 
specific,  or  the  60  most  embedded  nodes  in  the  WELL  integrated  curriculum  map, 
Luna  will  select  311  that  are  most  closely  related  to  his  goats. 

As  a  node  representing  a  skill  or  knowledge  descriptor  is  discussed  in  terms 
of  Mr.  Luna's  existing  skills  atid  proressional  objectives)  an  accompanying  text  field 
would  display  tin*  "market  demand  indicators  grid"  that  is  consulted  to  determine  the 
importance  of  the  particular  knowledge  description  to  the  various  WELL  Specialist 
employment  settings  (Hee  figure  IB). 

The  grid  would  indicate  the  importance  or  the  knowledge— rated  on  a  five- 
point  scale   to  three  major  employment  settings:  schools,  corporations,  and  agencies. 
The  scores  would  leptesent  ratings  by  employers  and  previous  graduates  or  the 
importance  of  the  particular  skill/knowledge  to  current  jobs.  This  information  would 
he  updated  hiiuuuiaHy  ami  wilt  be  based  on  questionnaires  distributed  to  WELL 
graduates  and  their  employers  The  nodes  of  interest  to  Mr.  Luna  would  be  those 
w  ith  a  "matket  demand  Indicators  grid '  score  or  greater  than  three  for  "school 
settings  "  These  ate  flagged  fin  future  reference  After  this  process  is  exhausted,  the 
complete  net  gi  ld '  showing  nil  flagged  nodes  and  their  interconnections  would  he 
viewed 

This  initial  academic  i-miuse|iitg  session  Would  conclude  with  the  scheduling 
of  a  follow  up  session  Mi  Luna  will  be  given  a  diskette  containing  the  WELL 
Hpei  lolls!  net  and  lnsli  in  led  on  how  to  use  it  at  the  program's  computer  lab  so  that 
be  iua>  work  with  the  net  Independently  to  explore  the  possible  options. 

In  the  second  session,  the  counselor  and  Mr.  Luna  would  discuss  the  result  or 
bin  work  with  rtie  WELL  Hperlallst  net  A  set  of  desired  competencies  will  be 
identified,  designed  to  balance  bin  interests  and  the  needs  of  the  job  market.  The 
next  step  would  be  to  view  the  options  for  'learning  activities"  that  are  attached  to 
tin*  selected  nodes  These  activities  would  includelecture,  independent  study 
options,  ami  internship  experienc  es  Mr.  Luna  selects  a  pattern  or  competencies  that 
aie  subsumed  within  several  three  unit  courses:  jf'syi  hintkfivnl  FntimlatumH, 
Sih  m/oj/iuif  hmmhUnnn,  (ivHtWi  h  Mvlhtxfa  ami  Inntruvtiotutl  DhiIh  and  Twhniqucn. 
These  courses  will  he  included  in  Mr.  Luna's  initial  educational  plan. 

A  perusal  of  I  he  mat  kef  demand  indicator  grid  for  the  school  settings 
unguesl s  Hint  knowledge  ol '"oiganiyatinual  improvement"  is  important  competency 
in  tlit'  opinion  ofemploveis  and  gniduatim    Mr.  Luna  and  the  counselor  agree  that 
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Figure  13*   "Market  Demand  Indicator  Grids**  accessible  through  selected 
"knowledge  descriptor"  nodes  would  provide  estimates  of  the 
importance  of  a  particular  knowledge  or  skill  element  in  three  types  of 
work  settings.  Ratings  displayed  are  Tor  "organisational  improvement" 
and  would  be  based  on  surveys  of  employers  and  graduates. 

the  independent  study  module  represented  in  the  map  is  a  desirable  mean*  of 
acquiring  this  competency.  In  addition,  Mr.  Luna  indicates  an  interest  in  pursuing 
an  independent  research  project  stemming  from  his  earlier  work  with  parental 
influence  on  Hispanic  children's  learning  successes.  He  and  the  counselor  decide  thai 
an  action-oriented  research  apprenticeship,  coordinated  with  a  faculty  member 
involved  in  this  area  of  research,  would  meet  this  need. 

At  the  close  of  the  session*  a  list  of  courses  and  learning  activities  would  he 
drawn  up,  providing  Mr.  Luna  with  an  initial  educational  plan.  He  will  be  then 
encouraged  to  consult  the  WtiLL  Specialist  net  again  and  to  initiate  a  follow-up 
counseling  session,  with  the  understanding  that  as  he  begins  to  implement  his  plan, 
his  new  experiences  and  interests  will  possibly  lend  to  revisions.  His  experience  in 
using  the  WELL  Specialist  net  and  his  access  to  information  regarding  the  available 
competencies,  learning  activities,  and  market  demand  indicators  will  make  him  an 
active  partner  in  this  dynamic  process  of  academic  advising. 
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Forward 

The  purpose  of  thti  paper  it  to  give  an  overview  of  the  most  recent  Educational 
Technology  Equipped  Classroom  (ETEC)  developed  at  the  8tate  University  of  New  York  at  Buffalo. 
EmphMts  wiH  be  pi«?ed  on  the  redesign  of  the  teaching  station  and  how  the  various  media  and 
support  technology  are  Incorporated  Into  the  custom  buHt  podium.  The  rationale  for  some  of  the 
changes  in  this  latest  ETEC  classroom,  based  on  assessment  of  earlier  classrooms  and  Input  from 
the  faculty,  are  also  diecu*aed. 

The  authors  had  the  privilege  of  presenting  a  paper,  Media  Equipped  Cteeerooms: 
Giving  Attention  Te  The  Teaching  Station,  at  the  1992  AECT  Conference  In  Washington, 
D  C.  This  current  paper  is  an  extension  and  further  refinement  of  technology  In  the  classroom. 
The  earlier  paper  dealt  with  our  Initial  Media  Equipped  Classrooms  and  the  design  of  our 
original  ETEC  teaching  station  for  the  School  of  Management.  Emphasis  was  ptwed  on  the 
rationale  for  change,  the  planning  process,  orientation  of  faculty,  assessment  ol  the  teaching 
station  as  well  as  extensive  CAD  drawings  and  cost  data.  If  Interested,  readers  can  obtain  a  copy 
of  this  earlier  paper  through  the  ERIC  Clearinghouse  on  Information  Resources  (ED  346  S41). 

Although  the  overall  reeponsibillty  for  planning  and  implementing  this  project  was 
assigned  to  the  presenters,  it  could  not  have  been  successfully  completed  without  the  assistance 
and  expertise  of  many  of  the  staff  in  Academic  Services  and  Operational  Support  Services,  two 
units  within  Computing  and  Information  Technology.  In  particular,  the  presenters  wish  to 
acknowledge  Dr.  Richard  Lesnlakt  Director  ol  Academic  Services,  Mar*  Greenfield,  Supervisor 
of  Media  Equipment  Services,  William  Maiman,  Project  Engineer,  Assistant  Engineers  Mark 
White  and  James  Crone,  and  Student-Assistants  Jeffrey  Oritsavage  (a  Civil  Engineering  student 
who  developed  the  CAD  drawings  and  a  ergonomic  analysis  of  the  teaching  station)  and  Aaron 
Somersteln  (a  School  of  Management  student  who  provide  general  protect  assistance  including 
configuring  the  teaching  station  PS/2  computer  hard  drive) 

Background 

State  University  of  New  York  at  Buffalo 

The  State  University  of  New  York  at  Buffalo  is  the  most  comprehensive  unit  of  the  64 
campus  SUNY  system,  enrolling  approximately  26,000  undergraduate  and  graduate  students  As 
one  of  four  university  centers,  Its  mandate  Is  to  concentrate  on  upper  division  and  graduate 
programs  and  research  It  offers  g3  undergraduate,  113  master's  and  83  doctorate  degree 
programs,  as  well  as  four  professional  degree  programs  and  14  graduate  certificates.  Faculties 
Include  not  only  the  traditions!  arts  and  sciences  divisions  of  Arts  and  Letters,  Natural  Sciences 
and  Mathematics,  and  Social  Sciences,  but  also  the  Undergraduate  College,  Millard  Fillmore 
College  (Evening  Division)  and  Schools  of  Architecture,  Dental  Medicine,  Education, 
Engineering,  Health  Related  Professions,  Library  Studies,  Law,  Medicine  and  Biomedical 
Sciences,  Nursing,  Pharmacy  and  Boclal  Work. 

These  diverse  programs  are  housed  on  two  separate  campuses.  The  older  South  Campus  In 
inn  City  of  Buffalo  has  been  refurbished  into  a  Health  Sciences  Center.  A  large  new  North 
Campus  was  recently  completed  in  the  suburb  of  Amherst,  approximately  five  miles  from  the 
Health  Sciences  campus. 


26 


Computing  and  Inform* Ion  Technology : 


During  the  Summer  of  1902,  a  major  organizational  cbanga  took  place  in  which  the 
former  Educational  Technology  Seivices,  along  wkh  eaveral  other  service  unto,  were  more 
ckaety  Integrated  into  Computing  and  Information  Technology.  Tht  reorganization  coneolldated 
abt  separate  unite  Into  four  distinct  unto  m  order  to  take  full  advantage  of  the  newer 
technologies;  especially,  computers,  networking,  communications  technology  and 
microcomputer  software.  This  realignment  of  staff  along  functional  commonalities  should 
position  the  orgmixatlon  to  provide  Improved  support  and  development  for  facHklos  such  as  our 
ETfC  classrooms  and  Instructional  support  labs,  m  which  the  convergence  of  computing  and  the 
more  traditional  educational  technologies  is  increasingly  evident.  Briefly,  these  four  unto  are 
described  as  follows: 

Acadamte  Strvlcsi,  Computing  ft  Information  Technology 

This  unit  focuses  on  Instructional  and  research  computing  support,  art  and  photographic 
servlcee,  media  resources,  network  uLer  support,  information  technology  centers  and  UB  micro 
computer  sales. 

Administrative  Computing  Services 

This  unit  focuses  on  computer  support  for  registration,  student  accounts,  financial  aid, 
housing,  financial  systems,  admissions  and  data  administration. 

Iachoicil  BuflMfl  Bmlcfla 

This  unit  focusss  on  software  support  of  mainframe,  UNIX  and  LAN  systems,  networking 
and  office  systems, 

Ogtmianil  BUBPQiL8ialcas 

This  unit  focuses  on  computer,  network  and  telephone  operations;  equipment  repair; 
telephone  services;  and,  facilities  and  systems  engineering  development  for  Computing  and 
Information  Technology 

In  the  design,  development  and  support  of  the  Educational  Technology  Equipped 
Classrooms  at  the  Slate  University  of  New  York  at  Buffalo,  expertise  and  support  is  drawn  from 
Academic  Services  (media  resources),  Technical  8upport  Services  (LAN  support)  and 
Operational  Support  Services  (design  and  installation,  aquipment  repair  and  maintenance). 


introduction  -  Educational  Technology  Equipped  Classrooms 

Over  the  last  four  years,  five  medium  size  classrooms  have  been  renovated  at  the  State 
University  of  New  York  at  Buffalo  for  the  purpose  of  incorporating  some  of  the  latest 
presentation  technology  The  first  three  classrooms  ware  referred  to  as  Media  Equipped 
Classrooms  (MEC).  They  were  not  only  designed  to  incorporate  both  video  and  computer  display, 
but  also  reflected  changes  In  the  mode  in  which  services  were  provided.  Faculty  teaching  in  a 
Media  Equipped  Classroom  would  have  immediate  access  to  a  variety  of  media  equipment  and 
support  technology  without  having  to  reserve  the  equipment  prior  to  their  class  session. 


ERLC 


27 


The  types  of  equipment  placed  In  these  MEC  classrooms  Included  an  overhead  projector, 
36mm  elide  projector(s),  10mm  motion  picture  projector,  video  tape  player  (1/2"  VHS 
and/or  3/4*  U-Matlc),  audio  cassette  tape  recorder/player,  sound  reinforcement,  laser 
jotrrter,  microcomputer,  video/data  monitor  or  CRY  projector,  projection  screens  and  custom 
designed  cutoinets  to  securely  store  this  equipment.  A  telephone  and  data  lines  were  also  installed 
In  each  ol  these  MEC  classrooms. 

though  welt  received  by  the  faculty  assigned  to  teach  in  these  Media  Equipped 
Classrooms,  theee  teWUee  had  some  inherent  pjoWems.  Faculty  had  to  open  up  the  storage 
Mtitfwt  (combination  lock),  remove  the  equipment,  set  It  up  and  then  at  the  conclusion  of  the 
dMS  session  property  secure  the  equipment  in  the  MEC  cabinets.  When  using  a  microcomputer, 
the  display  monitor  and  keyboard  were  pulled  out  of  the  cabinet  (the  CPU  unit  stayed  Inside  of 
the  cabinet)  and  placed  on  a  desk.  Not  only  were  there  a  mess  of  cables  to  contend  with,  proper 
lighting  at  the  tewhlng  station  was  always  a  problem  because  the  classroom  had  to  be  darkened 
sufficiently  In  order  to  display  data  through  the  QE  imager  310  projector  mounted  on  the 
celling.  These  MEC  classrooms,  although  a  vast  Improvement  over  the  traditional  classroom, 
were  not  as  the  staff  had  hoped  for,  nor  did  they  incorporate  the  latest  in  presentation 
technology. 

The  fourth  renovated  classroom,  currently  referred  to  as  an  Educational  Technology 
Equipped  Classroom  (ETEC),  incorporated  some  of  the  latest  presentation  technology.  It  was 
designed  to  support  the  curriculum  In  the  School  of  Management  and  was  ready  for  the  beginning 
of  the  1991  Fall  semester. 

This  ETEC  classroom  features  a  custom  designed  teaching  station  (modular  podium/ 
attached  side  cabinet)  that  houses  a  variety  of  electronic  presentation  systems  Including  an  IBM 
PS/2  microcomputer  (model  55  SX  which  supports  80386  applications),  a  Wolf  Vleuallzer  (a 
CCD  camera  platform  that  converts  transparencies,  hardcopy  and  smell  three-dimensional 
object*  to  video),  a  Navitar  VldeoMate  (converts  35mm  slides  to  video)  and  a  VHS  videotape 
player.  These  "electronic"  images  are  displayed  on  a  large  screen  with  a  G  E  410  Imager 
data/video  projector  mounted  on  the  celling.  Sound  reinforcement  (P  A,  system)  and  audio 
recording  are  controlled  from  the  teaching  station.  A  laser  pointer  is  also  available. 

Like  the  first  three  MEC  classrooms,  voice  and  data  lines  are  an  Integral  part  of  this 
Educational  Technology  Equipped  Classroom.  An  auto/dial  telephono  is  installed  In  the  teaching 
station  that  terminates  at  ths  Media  Equipment  Services  office.  In  the  event  of  an  equipment 
failure  or  an  operational  problem,  the  instructor  can  have  immediate  contact  with  a  trained 
media  specialist  for  assistance,  in  addition,  the  Instructor  can  place  calls  to  the  Computing 
Center  (DCA/Etftemet  connections),  School  of  Management  Computer  Laboratory,  Office  of 
Public  Safety  and  the  Physical  Plant.  A  data  line  is  also  prcvided  which  permits  the  instructor 
to  access  an  IBM  mainframe  computer,  network  servers,  or  numerous  other  connections  and 
services  provided  through  our  University  data  networks. 


A  unique  feature  of  this  ETEC  classroom  is  that  it  is  connected  to  the  School  of 
Management's  Local  Area  Network  'LAN).  When  the  IBM  PS/2  computer  is  turned  on,  it 
automatically  boots  up  into  the  School  of  Management's  LAN  system.  The  advantage  to  the 
instructors  teaching  in  this  classroom  is  that  they  can  incorporate  demonstrations  using  the 
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LAN's  software  and  data  IMS.  Space  is  limited  on  the  hard  drive  and  there  is  no  need  to  take 
valuable  cSass  time  to  load  a  software  program  into  the  hard  drive  of  the  IBM  computer. 

The  LAN  Is  wired  using  a  Ethernet  topology,  the  standard  chosen  by  the  Univeislty  for 
networking  computers.  The  Jacobs  110  ETEC  classroom  Is  one  of  42  IBM  PS/2  workstations 
connected  to  the  LAN  wfthm  the  School  of  Management.  The  network  uses  an  IBM  PS/2  Model  70 
personal  computer  as  a  file  aerver.  The  Ills  server's  120  MB  drive  storee  the  Novell  and  DOS 
operating  systems  aa  well  as  all  the  necessary  application  software  and  printing  utilities. 

At  the  workstation  level  ( e.g.,  the  Jacobs  110  ETEC  classroom),  the  ussr  ID  Is  the 
workstation.  Each  workstation  has  a  PROM  chip  and  an  Ethernet  board  to  make  the  connection  to 
the  file  cerver  when  the  work  station  Is  turned  on  and  to  Identify  that  particular  workstation. 
This  configuration  allows  all  processing  to  be  done  at  the  Jacobs  1 10  ETEC  workstation.  Using 
the  Novell  feature  of  distributed  processing,  the  burden  of  moat  of  the  Input/output  requests  is 
handled  by  the  memory  of  an  Individual  workstation  within  ths  network.  This  feature  prevents 
the  typical  slow  down  problem  many  networks  experience  when  the  server  must  uss  Its  own 
memory  to  manage  input  and  output  requests  from  all  the  other  workstations  on  the  LAN. 
Approximately  40K  of  the  memory  in  the  Jacobs  110  workstation  Is  used  for  the  network  shell. 
The  shell  interprets  each  request  made  by  the  user  and  directs  ths  request  to  either  the  file 
server  or  DOS,  depending  on  the  type  of  request 

At  the  conclusion  ol  the  1901  Fall  Semester  a  questionnaire  was  sent  to  all  faculty 
teaching  In  this  first  ETEC  classroom.  Out  ol  the  fifteen  questionnaires  sent  out,  fourteen  faculty 
responded.  In  general,  everyone  who  responded  was  quite  pleased  with  the  technology  made 
available  Yet,  there  were  criticisms.  8oms  were  dissatisfied  with  the  quality  of  the  Image  of  the 
Wolf  Vlsualizor.  Most  felt  that  the  current  placement  of  the  visuallzer  made  it  awkward  to  use. 
Many  asked  for  more  surface  spsce  on  the  teaching  station  to  layout  their  lecture  notes  and 
presentation  material!.  At  the  completion  of  the  1902  Spring  Semester  the  same  questionnaire 
was  sent  out  to  the  faculty  teaching  in  (his  ETEC  classroom  and  the  same  conclusions  were 
replicated.  Selective  follow-up  Interviews  were  also  conducted  to  assist  in  arriving  at  some 
possibls  solutions  to  these  criticisms. 

Many  of  the  School  of  Management  faculty  indicated  that  they  could  not  take  full  advantage 
ol  the  technology  unless  they  redesigned  their  courses  and  It  would  be  difficult  to  justify  the 
time  or  effort  It  would  take  unless  they  were  assured  of  being  assigned  to  an  ETEC  classroom  in 
ths  future.  Unfortunately,  the  majority  ol  the  laculty  assigned  to  teach  In  this  ETEC  classroom 
in  the  1991  Fall  8emester  were  not  assigned  to  this  classroom  for  the  1992  Spring  Semester.  A 
fair  amount  ol  disappointment  was  expressed  by  the  School  ol  Management  laculty.  A  positive 
outcome  of  this  disappointment  was  that  endowment  funding  was  Identified  by  the  School  of 
Management  to  develop  a  second  ETEC  classroom. 

The  planning  of  this  second  ETEC  classroom  for  the  School  of  Management  provided  an 
opportunity  to  alter  the  design  of  the  teaching  station  and  to  make  appropriate  improvements 
where  feasible.  The  remainder  of  this  paper  will  discuss  this  new  ETEC  classroom  and  the 
improvements  made,  based  on  (acuity  feedback. 
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Educational  Technology  Equipped  Classroom  -  Re-Design 


This  second  Educational  Technology  Equipped  Classroom  (ETEC)  is  located  In  112  Jacobs 
Hall  and  replicates  the  same  presentation  technology  as  the  first  ETEC  classroom  located  in  1 10 
Jacobs  Hall.  This  new  classroom  is  slightly  smaller  in  setting  capacity  (56  students)  and  haa 
some  subtle,  yet  significant  Improvements  in  the  design  of  the  teaching  station  as  weM  as  some 
Improvements  In  the  processing  of  video  signals  and  image  quality.  It  should  also  be  noted  thai 
the  changes  Incorporated  into  the  teaching  station  of  this  second  ETEC  classroom  were  also 
retrofitted  into  the  earlier  teaching  station  in  110  Jacobs  Hall.  This  retrofit  could  be 
MOompNshed  at  minimal  expense  due  its  modular  construction.  Also,  It  was  felt  that  both  of 
these  ETEC  classrooms  should  be  replicated  as  closely  as  possible  because  School  of  Management 
faculty  could  be  assigned  to  teach  in  either  classroom  depending  on  course  scheduling  end  class 
size. 

Planning  and  Implementation: 

Like  the  earlier  Jacobs  110  ETEC  classroom,  faculty  input  was  sought  through 
questionnaires  and  meetings.  After  their  initial  input,  several  informal  review  sessions  were 
held  with  the  faculty  to  Keep  them  appraised  of  developments  and  to  gather  appropriate  feedback 
Faculty  Input  is  see  as  absolutely  essential  if  these  ETEC  classrooms  are  to  be  accepted  and  fully 
utilized. 

A  classroom  assessment  was  carried  out  by  the  project  engineer  (Operational  Support 
Services  unN)  utilizing  criteria  developed  for  the  earlier  MEC  and  ETEC  classrooms.  The 
assessment  would  determine  what  problems,  if  any,  existed  that  would  prevent  or  hinder  the 
proper  utilization  of  media  equipment  and  other  kinds  of  classroom  support  technology.  Issues 
such  as  viewing  distance,  site  lines,  window  blocking,  zoned  lighting,  acoustics,  ambient  noise, 
eloctrical  outlets,  etc.,  all  affect  proper  utilization. 

The  classroom  selected  (Jacob  1 12)  has  a  student  seating  capacity  of  55  and  met  all  of 
the  basic  criteria  An  added  feature  was  that  this  was  a  tiered  classroom  with  no  windows.  Zoned 
incandescent  lighting  was  already  in  place  as  well  as  an  electric  wall  screen.  CAD  drawings 
depicting  room  layout  and  site  lines  are  In  the  Appendix. 

The  rehab  of  the  classroom  took  place  over  the  1992  Summer  recess  and  was  carried  out 
by  both  the  trades  staff  of  the  Physical  Plant  and  the  Operational  Support  Services  staff.  The 
teaching  station  (podium  and  side  modular  cabinet)  was  permanently  fixed  in  place  and  conduit 
run  for  electrical  service  and  control  circuits.  All  conduit  was  either  trenched  in  the  floor  or 
run  In  the  celling.  A  distributed  speaker  system  was  installed  In  the  ceiling  as  well.  All 
component  installation,  control  circuits  and  wiring  was  handled  by  the  Operational  Support 
Services  staff. 

The  overall  project  took  less  than  six  months  from  start  to  finish.  Approximately  three 
months  was  spent  in  planning,  design  and  procurement  of  equipment.  Classroom  rehab  and 
installation  of  the  equipment  took  less  than  three  months.  Everything  was  completed  in  time  for 
the  start  of  the  1992  Fall  semester. 
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The  Teaching  Station: 


The  teaching  elation  consists  of  a  custom-built  podium  and  an  attached  modular  cabinet, 
like  the  earlier  Jacobs  110  ETEC  classroom,  CAD  drawings  and  specifications  were  developed  by 
the  Operational  Support  Services  staff  and  the  unit  was  custom  built  by  Rabco  Design,  Inc.  of 
Buffalo,  New  York. 

The  significant  changes  between  the  earlier  teaching  station  and  the  current  teaching 
station  are  in  the  placement  of  the  Wolf  Visualize*  and  its  controls,  and  the  placement  of  the 
podium  surface  lighting.  The  earlier  design  placed  the  WoH  Vieuallxer  on  a  pull-out  shelf  and  It 
was  accessed  from  behind  the  modular  cabinet.  When  the  vieuallzer  was  in  use,  the  Inetructor 
had  to  stand  away  from  the  vieuallzer  and  behind  the  podium  which  made  It  awkward  to  use.  The 
instructor  found  it  difficult  to  use  the  surface  of  the  modular  cabinet  for  presentation  materials 
and  lecture  notes  and  the  vlsualizer  controls  were  difficult  to  use  because  of  placement  and 
design. 

The  Wolf  Vlsualizer  is  now  located  on  a  drawer  unit  which  pulls  out  to  the  side  of  the 
modular  cabinet.  A  more  simplified  control  pad  was  custom  designed  and  placed  on  the  top 
surface  of  the  modular  cabinet.  The  podium  lighting  was  changed  from  ftorescent  to  Incandescent 
and  raised  a  few  inches  to  better  Illuminate  the  podium  work  surface  and  reduce  glare  on  the 
Instructors  monitor  screen.  The  height  of  the  modular  cabinet  was  also  raised  a  few  Inches  to 
make  the  platform  of  the  vlsualizer  more  comfortable  to  use. 

Other  advancements  were  made  In  the  electronic  circuitry  and  signal  processing.  In  the 
earlier  ETEC  design,  only  data  could  be  displayed  on  the  podium  monitor.  Through  a  scan 
doubler,  both  data  and  video  are  now  displayed  on  the  podium  monitor.  This  allows  the  Instructor 
to  view  the  same  Image  on  the  teaching  station  monitor  that  is  displayed  In  the  classroom 
through  the  video/data  projector.  Alternative  approaches  were  considered  to  accomplish  this 
design  objective  (a  multi-scan  monitor,  IBM  UltlMedia  M-Motlon  Video,  IBM  PS/2  TV).  Each  of 
these  were  rejected  for  a  variety  of  reasons  which  are  too  involved  to  discuss  in  this  paper.  The 
use  of  a  scan  doubler  for  all  video  signals  also  brought  Improved  Image  quality  to  the  front 
classroom  seats  by  eliminating  visible  scan  lines  In  the  projected  Image. 

An  ongoing  ETEC  design  objective  is  to  continue  to  incorporate  improvements  which  will 
reeult  in  making  the  computing  system(s)  more  responsive  to  Instructor  commands  (I.e.,  faster 
access  to  applications,  Increased  speed  In  Interactive  sessions,  etc.).  The  PS/2  computer  was 
upgraded  from  30  to  80  MB  hard  drive  and  the  RAM  was  increased  from  2  to  4  MB  In  order  to 
accommodate  new  software  and  allow  the  system  to  be  more  responsive.  By  also  going  from  a  DCA 
to  an  Ethernet  data  network  connection,  the  data  throughput  rate  was  significantly  expanded 
(9.6  Kbps  to  2-3  Mbps). 

The  podium  and  side  cabinet  are  constructed  and  finished  as  separate,  stand-alone  Kerns 
with  3/4"  particle  board  construction.  The  Interior  and  exterior  are  finished  with  Formica 
brand  laminate.  The  podium  keyboard  shelf  and  the  pull-out  drawer  and  one  shelf  In  the  side 
cabinet  are  on  slide-out  rails.  All  other  shelving  Is  adjustable.  The  three  doors  (one  cabinet 
door  and  two  podium  doors)  are  fitted  with  hinges  allowing  doors  to  be  opened  a  full  270 
degrees.  The  doors  are  opened  with  matching  combination  looks. 
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The  top  surface  of  the  podium  contains  a  flush  mounted  monitor  and  the  system's  control 
panel.  When  the  top  door  of  the  podium  is  opened  the  keyboard  shetf  can  be  puMed  out  The  auto 
dial  telephone,  VHS  videotape  player  and  computer  (CPU)  are  also  accessible  in  this 
compartment.  The  lower  door  Is  not  accessible  by  the  Instructor.  This  compartment  contains 
control  and  security  systems. 

The  top  surface  of  the  redesigned  side  cabinet  now  serves  as  a  table  top  for  instructor 
notes  and  preeemation  materials.  When  the  door  of  the  side  cabinet  is  opened,  the  drawer  unit 
tor  the  Wolf  VlsuaHzer  can  be  unlatched  and  pulled  out.  The  arm  of  the  visuaMzer  la  then 
extended  ml  the  unit  Is  ready  tor  use.  Below  the  visualizer  unit  Is  a  Navitar  slide  to  video 
projector  that  is  on  a  pull  out  shelf  for  ease  of  loading  the  carousel  tray  and  turning  the 
projector  on.  Sufficient  storage  space  exits  below  the  sHde  projector  for  the  microphone,  tape 
recorder  and  laser  pointer. 

The  flush  mounted  panel  on  the  top  surface  of  the  podium  controls  two  podium  lights, 
brightness  and  contrast  for  the  computer  monitor,  audio  level  for  the  VHS  player  and  for  an 
auxiliary  input,  power  and  mute  switch  for  the  celling  mounted  CRT  projector,  forward/ 
reverse  switch  for  the  slide  to  video  protector  and  six  switches  to  control  the  Input  to  the  CRT 
projector  -  Visualizer/VHS  player/Slides/PS-2  Computer/RGB  1/RQB2.  Auxiliary  input 
Jacks  are  located  on  the  top  of  the  side  cabinet,  two  ROB  inputs,  two  1/4"  phone  lacks  for  audio 
taping  and/or  playback,  and  two  microphone  inputs.  An  AC  duplex  plug  is  located  at  the  base  of 
the  side  cabinet. 

An  added  feature  for  ease  of  instructor  utllizatkr  Is  that  all  components,  with  the 
exception  of  the  Navitar  slide  projector,  are  automatically  turned  on  when  the  upper  podium 
door  or  the  side  cabinet  door  are  opened,  Both  doors  are  wired  to  electrical  Interlock  switches. 
There  is  no  need  to  turn  on  individual  equipment  off/on  switches.  Due  to  the  noise  of  the  cooling 
fan  on  the  Navitar  slide  projector,  it  was  felt  that  this  piece  of  equipment  could  be  turned  on 
only  when  used  for  slide  presentations 

The  audio  amplifier  for  this  ETEC  classroom  is  not  installed  in  the  teaching  station.  It  is 
recessed  In  a  classroom  wall  and  not  accessible  to  the  instructor.  Audio  levels  are  pre  set  and 
the  system  is  equalized  for  the  classroom  environment.  The  controls  for  the  zoned  incandescent 
lighting  are  on  rheostats  (each  bank  of  lights  are  on  Its  separate  rheostat).  Both  the  lighting 
controls  and  the  switch  for  the  projection  screen  are  located  on  the  wall  directly  behind  the 
teaching  station  and  within  easy  reach  of  the  instructor. 

The  Appendix  contains  CAD  drawings  of  the  tsachtng  station,  a  bid  specification  sent  to 
vendors  for  the  construction  of  the  teaching  station,  and  a  fist  of  equipment  and  components  and 
their  cost.  Expenditures  for  equipment  and  components  came  to  $38,795.00  for  the  Jacobs 
112  ETEC  classroom.  The  classroom  rehab  costs  from  the  Physical  Plant  (materials  and  labor) 
came  to  $4,994.00. 

Access  and  Security: 

The  proper  security  of  the  equipment  is  an  integral  part  of  these  Educational  Technology 
Equipped  Classrooms.  A  system  was  devised  that  would  help  protect  the  equipment  from 
vandalism,  theft  and  unauthorized  removal;  yet,  allow  faculty  designated  to  teach  in  these 
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classrooms  ••••  of  access  Instructor*  navs  access  to  the  equipment  and  systems  by  opening  mo 
upper  door  on  ths  podium  end  the  door  on  the  aide  cabinet,  Each  door  has  a  prog/ammable 
combination  lock  sat  op  with  ttv  same  combination.  The  oomblnationa  art  changed  one*  a  y*ar. 
Electronic  sensor  devtoas  are  faataned  on  tome  various  components  (CRT  projector,  computer, 
vlsualier,  etc.)  and  W  someone  attempted  to  remove  the  equipment  without  proper 
authorization  an  alarm  would  go  off.  Public  Safety  would  automatically  be  alerted  through  a 
telephone  lint.  As  an  added  security  precaution,  these  classrooms  are  locked  after  the  last 
scheduled  class  of  the  day  (around  1000  pm)  and  are  opened  Just  prior  to  the  start  of  the  first 
oiaes  session  (1:00  am).  These  classroom!  are  kept  locked  on  weekends  (after  scheduled 
Saturday  morning  classes),  holidays  and  during  aemeeter  breaks. 


Faculty  Orientation: 

Faculty  orientation  is  carried  out  by  the  Associate  Director  -  Media  Resources  and  his 
atari  prior  to  the  atari  of  classes  and  during  the  first  week.  A  list  of  facuky  assigned  to  teach  In 
these  ETEC  classrooms  is  obtained  through  the  School  of  Management.  A  letter  and  handout  is  sent 
to  each  faculty  member  explaining  the  ETEC  program  and  asking  them  to  attend  one  of  aeveral 
scheduled  orientation  sessions  or  to  call  and  make  an  Individual  appointment. 

The  on -sits  orientation  takes  about  one  hour.  The  different  types  of  media  and  classroom 
support  technology  are  demonstrated.  Access  to  the  teaching  station,  calling  for  staff  assistance 
and  security  procedures  are  covered  as  well.  Discussion  is  encouraged  and  men  the  remainder  of 
the  orientation  session  Is  spent  with  hands-on.  Each  participant  has  an  opportunity  to  open  the 
doors  (combination  lock)  to  the  podium  and  attached  cabinet,  set-up  the  equipment,  practice 
using  the  equipment  and  then  properly  secure  the  equipment.  Upon  conclusion  of  the  orientation 
session  faculty  are  given  a  laminated,  wallet  sire  card  with  the  combination  to  the  locks.  Each 
ETEC  classroom  is  set-up  with  a  different  set  of  combinations.  A  Media  Equipment  Services  atari 
member  Is  also  available  at  the  beginning  0f  the  semester  in  the  event  that  an  instructor  needs 
some  assistancs. 

Concluding  Remarks 

The  new  Jacobs  1 12  ETEC  classroom  has  been  In  use  for  one  full  semester.  Although 
there  has  been  no  formel  assessment  of  this  nsw  facility  through  a  survey  Instrument,  Informal 
feedback  points  out  that  this  classroom  has  been  well  received.  Faculty  are  quite  pleased  with 
the  placement  of  the  Wolf  Vlsualizer  and  the  location  of  the  custom  control  pad.  The  ability  to 
display  both  data  and  video  on  the  podium  monitor  has  also  proved  to  be  an  asset.  Faculty  have 
also  remarked  that  the  quality  of  the  projected  image  (through  the  use  of  the  scan  doubler) 
appears  to  be  better  as  well. 

During  the  Winter  break,  an  older  Media  Equipped  Classroom  will  be  upgraded  Into  a 
Educational  Technology  Equipped  Classroom  with  ths  red*slgn*d  teaching  station.  This  third  ETEC 
classroom  will  be  used  primarily  by  the  Department  of  Computer  Science.  In  addition  to  the 
teaching  station,  an  older  Macintosh  llsi  will  be  upgraded  to  a  Macintosh  llvx  with  an  Internal 
CD  ROM  and  the  QE  310  Imager  projector  will  be  upgraded  to  a  QE  414  Imager  projector.  A  Wolf 
Vlsualizer  will  be  added  as  well  This  ETEC  classroom  will  be  completed  In  time  for  the  start  of 
ths  1903  Spring  Someater. 
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Ntw  Directions: 


A  Technology  in  the  Classroom"  working  group  hat  teen  recently  formed  within 
Academic  Services,  Computing  and  Information  Technology  to  further  explore  how  technology 
can  beet  be  developed  and  utilized  In  the  classroom  environment.  This  group  wNI  make 
recommendations  for  further  development  of  Educational  Technology  f  quipped  Classrooms;  and, 
look  at  ways  m  which  some  of  the  emerging  technologies  such  as  CO  ROM  and  Digital  Video 
Interactive  (DVI)  affect  the  classroom  environment.  This  group,  working  closely  with  faculty, 
wH  also  determine  directions  for  development  of  additional  ETEC  software  options;  either  to  tie 
made  available  locally  at  the  teaching  station,  or  remotely,  through  the  campus  dale  network. 

Another  challenge  of  this  working  group  will  be  to  explore  ways  m  which  the  ETEC 
concept  can  be  effectively  Incorporated  in  large  lecture  haNs.  A  Natural  Sciences  and 
Mathematics  lecture  hall  complex  Is  currently  under  construction  (five  tiered  lecture  hails 
ranging  from  100  seats  to  366  seats  and  five  00  seat  classrooms)  and  funding  has  been  allocated 
for  classroom  technology.  Plans  are  also  underway  to  convert  existing  lecture  halts  to  the  ETEC 
service  model  as  well. 

The  Appendix: 

A  variety  of  information  is  contained  in  the  Appendix.  A  number  of  CAD  drawings  are 
offered  showing  the  teaching  station,  room  layout,  and  systems  interconnection.  The  Appendix 
also  contains  information  regarding  the  cost  of  the  equipment  and  components,  a  copy  of  the 
purchase  requisition  and  bid  specifications  for  the  custom-built  teaching  station,  a  list  of  the 
software  currently  used  by  the  School  of  Management,  and  computer  menu  screens. 

Conclusion: 

In  conclusion,  the  purpose  of  this  presentation  was  to  give  an  overview  of  the  latest 
Educational  Technology  Equipped  Classroom  (ETEC)  at  the  State  University  of  New  York  at 
Buffalo  with  specific  reference  to  the  re-design  of  the  teaching  station  based  on  faculty  feedback. 
Although  this  latest  ETEC  classroom  has  onty  been  in  service  for  one  semester  all  indications 
are  that  it  is  been  well  received  by  the  faculty  assigned  to  teach  in  this  facility. 
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Apptndlx 


1  CAD  Drawing  i>f  Teaching  Station:  Front  View  (docrt  cloeed) 

2  CAD  Drawing  of  Teaching  Station.  Front  View  (vlsualuer  extended) 

3.  CAD  Drawing  of  Teaching  Gallon  Bide  Viaw  (original  design  ve.  new  design) 

4.  CAD  Drawing  of  Taaching  Slallon:  Bide  Vlaw  (dimensions) 

6.  CAD  Drawing  of  Taaching  Station  Podium  Front  Viaw  (dimensions) 
0  CAD  Drawing  of  Taaching  Station  Cabinet  Bide  Viaw  (dimensions) 

7.  CAD  Drawing  of  Jacobs  1 12  Room  Layout 

a.  CAD  Drawing  of  Teaching  Station  Control  Panel 

B.  CAD  Drawing  of  Teaching  Station  Remote  Control  for  Wolf  Visualizer 

10.  CAD  Drawing  of  Teaching  Station  Podium  and  Cabinet  Work  Surfaces:  Top  View 

1 1 .  CAD  Drawing  of  System  Interconnection  Diagram 

12.  CAD  Drawing  of  ETEC  Network  Connections 

13.  ETEC  Computer  Menu  Screens  (main  menu,  software  applications,  Telnet  and  School  of 

Management  software) 

14.  Jacobs  112  Equipment/Cost  Data 

15.  Jacobs  112  Teaching  Station  Purchase  Requisition  and  Bid  Specification 
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DESCRIPTION: 
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BUIUHNG/CUENT: 
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Jacobs  112  Equipment/Cost  Data 


Equipment  and  Components 

Amount 

1. 

Teaching  Station 

$  3,800.00 

2. 

G.E.  #414  Video/Data  Projector 

8,790.00 

3. 

Video/Data  Projector  Ceiling  Bracket 

537.17 

4. 

Switcher,  Distribution  Amp,  Cables 

1,476.50 

5. 

S-Video  Components 

819.62 

6. 

Inline  IN1222  Scan  Doubler 

2,436.00 

7. 

PS/2-55SX  Computer,  8513  Monitor 

2,337.00 

8. 

G.E.  Wolfe  Visualizer 

6,870.00 

9. 

Panasonic  AG-1000B  Video  Cassette  Player 

506.78 

■i  n 
iU. 

Noviiar  viaeomaie  iuuu 

o  ctm  nn 

11. 

Sound  System  Items 

4,397.60 

12. 

Laser-X  LDP-5AV  Pointer 

410.00 

13. 

Auto  Dial  Telephone 

170.03 

14. 

Electronic  Alarm  System  and  Accessories 

192.17 

15. 

Projection  Screen  Fabric  and  Tension  Bar 

178.76 

16. 

Cable  and  Custom  Built  Items 

1,798.00 

17. 

Electrical  and  Miscellaneous  Supplies 

471.20 

Total       $  38,794.83 
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REQUISITION 

N?  949488  S 


PURCHASE  REQUISITION 

STATE  UNIVERSITY  OF  NEW  YORK 

AT 
BUFFALO 


PURCHASE  OROfeR  NUMBER 


VENDOR      RABCO  DESIGN,  INC. 


REOUiSi  HON  DATE  


STREET. 


8925  SHERIDAN  DRIVE 


STATE  CONTRACT  # . 


GROUP  # . 


CITY  &  STATE  CIARENCE,  N.Y.  14031,  631-3010 
FEDERAL  I.Q  #    


ACCOUNT  NUMBER. 


THIS  IS  NOT  A  VENDOR'S  ORDER 

NSTIHiCnONS: 

1  MAKE  SEPARATE  REQUISITION  FOR  EACH  VENDOR  AND  COMMODITY  GROUP 

2  ATTACH  ALL  LETTERS.  QUOTATIONS.  AND  OTHER  APPLICABLE  PAPERS  TO  THIS 
REQUISITION 

a  FORWkRO  THIS  REQUISITION  TO  THE  PURCHASING  AGENT 

a  USE  DOUBLE  SPACE  ON  ITEM  ENTRIES  


capihtvSCH,  OF  MANAGEMENT,  JACOBS  112  PROJ • 

DEPARTMENT  EDUCATIONAL  TECHNOLOGY  SERVICES 
ATTN:  WM.  MAIMAN 

SHIP  TO  BUILDING  WENDE         ROOM  NO.  318  


REQUESTED  DELIVERY. 


6/17/92 


fob 


□  DESTINATION 
□ 


SHIPPING 
PONT 


SHIP  VIA 


TERMS 


BUYER 


Mm  No 


D»cnpt«o  d  Mafrrtai/Service 


Quantity 


AUDIO  VISUAL  INSTRUCTOR  STATION 

CONSTRUCTION  OF  CUSTOM  INSTRUCTOR  STATION  CON- 
SISTING OF  SEPARATE  PODIUM  AND  CABINET  SECTIONS 
FOR  JACOBS  112  BASED  ON  DESIGN  OF  JACOBS  110 
AND  FEATURING  VARIOUS  CHANGES •     ALL  CABINETRY 
HARDWARE  TO  BE  PROVIDED  UNDER  THIS  ORDER  MUST 
BE  REVIEWED  AND  APPROVED  BY  UNIVERSITY  PROJECT 
COORDINATOR,     EQUIPMENT  ITEMS  AND  CONTROL  PANELS 
REFERENCED  IN  THE  ENCLOSED  ATTACHMENTS  ARE  NOT 
TO  BE  CONSIDERED  PART  OF  THIS  ORDER,     ALSO  TO 
BE  PROVIDED  UNDER  THIS  ORDER  ARE  FOUR  (4)  FULL 
WIDTH  ADJUSTABLE  SHELVES  AS  ACCESSORY  ITEMS, 
ALL  DRAWINGS  AND  CONSTRUCTION  NOTES  ENCLOSED 
AS  ATTACHMENTS  TO  THIS  ORDER  ARE  FOR  REFERENCE 
PURPOSES  ONLY.     CONTRACTOR  IS  REQUIRED  TO  COMPLETELY 
REVIEW  AND  ASCERTAIN  ALL  CONSTRUCTION  REQUIRE- 
MENTS AND  DIMENSIONS  WITH  UNIVERSITY  PROJECT 
COORDINATOR  PRIOR  TO  BEGINNING  CONSTRUCTION. 
PAYMENT  PLAN;     A  PARTIAL  PAYMENT  PLAN  CONSIST- 
ING OF  60%  PAYMENT  AFTER  COMPLETION  OF  CABINET 
(WITH  LAMINATE)  AND  REMAINDER  PAYABLE  AFTER  COMPLETE 
INSTALLATION  IN  JACOBS  112  CLASSROOM, 

PLEASE  NOTIFY  WM.  MAIMAN  AT  831-3761  IF  THERE 
WILL  BE  A  DELAY  IN  SHIPMENT  OF  THIS  ITEM(S). 


Unit 


Prce 


Amount 


EA 


3,800 


00 


PLEASE  CHECK  APPROPRIATE  BOX /BOXES  Q  HANDICAPPED 

□  MINORITY-OWNED  BUSINESS    □  SMALL  BUSINESS  (UNDER  100  EMPLOYEES)     □  WOMEN-OWNED  BUSINESS 


SOLICITED  MBE  . 


WBE  . 


IN  ACCORDANCE  WITH       ®  TELEPHONE  FAX      quotaTiON       BOB  BARONE 
□  WRITTEN 


DATED 


4/16/92 


OUR 
REF 
NO 


PO/Cootract 


Line 


Act 


Amount 


Dept 


Cost  Center 


Var 


Obftcl 


PUNDS  AVAILABLE  BY 


SUNY  ACCOUNT 


SUB 


OBJECT 


50 
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White:  Purchasing  Copy 
Pink:  Requltitlontr  •  DtUch  and  keep 


REQUESTED  BY 


TELEPHONE  NO 


ai  iTHO«t7rn  SIGNATURE 


Custom  Instructor  Station  Construction  Notes 


1 .  Podium  and  Cabinet  to  be  constructed  and  finished  as  separate,  standalone  items. 

2.  Podium  and  Cabinet  to  be  3/4"  heavy-duty  industrial  grade-dense  particle  board 
construction;  finished  a!!  exterior  and  inside  sides  with  Formica  brand  #118  Finnish 
Oak  (or  equal)  laminate. 

3.  Podium  keyboard  shelf,  and  top  two  Cabinet  shelves,'  to  be  on  slide-out  rails  capable 
of  supporting  150  lbs.  All  other  shelving  to  be  adjustable  (32mm  system),  and 
perforated  to  facilitate  airflow.  Shelf  cutouts  required  for  wiring  access. 

4.  All  doors  fitted  with  3  hinges  allowing  doors  to  be  opened  a  full  270  degrees.  All 
doors  to  also  include  matching  combination  locks.  Locks  to  match  type  installed  in 
Jacobs  110. 

5.  Podium  monitor  compartment  to  be  constructed  to  accommodate  either  an  IBM 
8513  or  IBM  8515  monitor.  A  monitor  mounting  configuration  is  required  which  will 
accommodate  either  monitor  model,  and  allow  monitor  positioning  adjustment. 
Monitor  mounting  configuration  to  match  existing  Jacobs  110  configuration. 

6.  Podium  work  surface  to  be  constructed  with  flush-mounted  safety  glass  panel  over 
the  monitor  areas;  and,  a  surface  cutout  for  a  flush-mounted  system(s)  control 
panel.  Cabinet  work  surface  to  be  constructed  with  a  surface  cutout  for  a  flush- 
mounted  system(s)  control  panel. 

7.  Provisions  must  be  made  in  the  construction  of  both  the  Podium  and  Cabinet 
sections  to  permit  vertical  airflow  and  equipment  ventilation.  Provisions  also  must 
be  made  to  allow  for  the  mounting  of  recessed  Wiremold  type  electrical  boxes. 

8.  Bottom  panels  in  both  the  Podium  and  Cabinet  constructed  so  as  to  be  removable, 
or  to  provide  access  to  the  base  area  for  ease  in  connection  of  system(s)  wiring. 

9.  Slide-out  platform  to  be  provided  for  Campus  Telephone. 

10.  Cabinet  connector  panel  constructed  so  as  to  be  removable. 

1 1 .  Separate  2"  x  4"  (nominal)  floor  mounting  frame,  to  be  provided  for  both  the  Podium 
and  Cabinet  to  facilitate  installation  and  reinforcement  for  anchoring  hardware. 

12.  Progress  Meetings  shall  be  scheduled  with  the  Projepf, Coordinator,  wm.  Maiman, 
831-3761,  to  review  progress  and  allow  for  custom  fitting  of  equipment. 
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FOUCAULT  AND  DISCIPLINARY  TECIINOUXIY 

"Where  the  sun  lights  up  the  swift  joys  of  deprived  animals/1 

•.•Baudelaire 


Crisis  of  representation  ("Let  X  equal  X".  Laurie  Anderson),  crisis  of 
authority  ("Your  gaze  stops  on  the  side  of  my  face."  Barbara 
Krugar),  crisis  of  subjectivity  (Phonographs  of  Cindy  Sherman 
dressed  up  as  many  different  women  subjects.  Who  is  the  real  Cindy 
Sherman?).  Politics  of  the  observed,  politics  of  the  observer  (Van 
Maanen,  1988;  Lather,  1991).  The  post-modern  condition  (l.yotard, 
1984  )  necessitates  that  what  claims  to  be  real,  true,  universal,  and 
timeless,  may  in  fact  be  partial,  time/ place  bound,  incomplete  and 
socially  constructed  (Toulmin  1990,  Rorty,  1989).  Kuhn  writes 
about  "paradigm  shifts",  Rorty  about  "historical  contingencies, 
I'burault  about  "epistemes"  and  discourses,  and  Wittgenstein  about 
"language  games".  As  we  draw  to  the  end  of  the  20th  century  all 
that  was  understood  to  be  true  is  up  for  question.  Rather  than 
seeking  Truth,  we  are  now  encouraged  to  seek  little  truths  which  are 
situational!}'  appropriate. 

1  or  Michel  I bucault,  it  is  important  that  we  question  every  thing, 
including  law,  science,  religion,  and  Western  philosophy.  An  in mu 
stance,  disciplined  thought,  and  practical  wisdom  helps  people 
invent  themselves,  rather  than  being  concerned  with  finding 
themselves,  lbucault  believes  in  reason,  but  he  believes  that  it  is 
important  to  place  limits  on  reason  and  realize  that  there  are  times 
when  humor,  the  imagination  and  the  acceptance  of  contradictions 
( o-existing  can  help  people  develop  a  health)  attitude  about  living  in 
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this  world,  Kather  than  railing  for  unifying  around  a  project,  such 
as  llahermas's  emancipatory  project,  Foucault  suggests  accepting  a 
sex  i(*t\  ol  dilierence  and  the  developing  of  a  ethos  ("Lile  as  art." 
Baudelairfc).  lor  I:ou<  ault,  rather  than  being  concerned  with  using 
language  for  converging  on  a  concept  of  community,  language 
should  be  used  to  open  up  possibilities. 

I'ou<  ault  works  with  what  he  calls  methods  of  archeology  and 
genealogy:  he  studies  current  conditions,  tries  to  determine  what 
has  allowed  these  conditions  to  develop,  and  then  opens  up 
possibilities  of  new  ways  of  looking  at  these  issues.  A  locus  in  his 
study  is  the  relationship  between  power  and  knowledge,  lor 
lourault,  those  who  seek  and  maintain  power  over  others  Irequentl) 
use  three  disciplinary  technologies  which  shape  people  into 
accomodating,  docile  bodies: 

a.  Surveillance  -  ordering  bodies  in  space  and  time  so 
that  can  can  easily  be  observed  or  think  they  are  being 
observed. 

b.  The  Examination  -  testing  people  so  that  they  can 
be  compared  with  others  in  a  group,  while  also  be 
further  individualized. 

c.  Normalizing  judgement  -  convincing  people  of 
correct  action,  thoughts,  and  truth. 

Some  questions  concerning  educational  technology  which  emeige 
from  my  reading  of  loucault  include: 

a.  Should  we,  as  educational  technologists,  be  concerned  with 
how  we  position  the  whole  bodies  of  the  teachers  and  students/ 

b.  How  can  materials  which  aim  at  reliability,  replicability, 
algorithmic  decision  making,  control  and  quantitative  evaluative 
measures  develop  the  critical  thinking  capabilities  of  a  person? 

c.  Are  there  appropriate  sites  and  non 'sites  for  educational 
technology?  If  so,  how  do  we  determine  appropriateness? 

d.  What  is  our  theory*  of  instruction?  Do  we  aim  to  teach 
students  what  they  should  learn,  or  do  we  aim  to  teach  students  how 
to  learn?  vv 

e.  Is  one^a  teachers  responsibilities  making  students 
comfortable  with  receiving  directions  and  trusting  information 
which  conies  from  technology? 
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Paradigms  Re framed: 
Constructivist,  Post-industrial, 
Modern  or  Postmodern  Educational  Technology  ? 

by  Denis  Hlynka 
Professor 
University  of  Manitoba 
e-mai 1 :  dhlynka@ccm. umanitoba .ca 


There  is  some  confusion  and  some  reluctance  to  accept  the 
term  "postmodern"  within  mainstream  educational  technology. 
Opponents  argue  that  it  is  too  vague  a  term,  too  slippery, 
too  literary,  too  negative,  or  just  plain  inappropriate. 
Proponents  argue  otherwise. 

"Modernity"  reflects  an  almost  scientistic  belief  in  the 
progress  of  the  twentieth  century.  Yet  others  argue  that 
modern i ty  did  not  deliver  all  that  it  promi sed .  That 
questioning  attitude  towards  the  thrust  of  modernism  is  a 
post-modern  view.  ; 

There  seems  to  be  less  concern  less  trouble  when  the  noun 
is  "industrialization."  We  have,  likewise  moved  beyond  the 
"industrial"  revolution,  beyond  our  fascination  with  the 
machine  as  saviour,  towards  what  has  been  dubbed  a  "post- 
industrial"  world  view,  one  in  which  the  benefits  of 
industrialization  are  accepted,  but  critically. 

Psychology,  for  many  the  root  metaphor  for  educational 
technology,  has  also  made  a  similar  move.  The  Behayiorist 
model  led  to  the  cognitivist.  Today,  constructivism  is  seen 
as  a  next  step  in  the  cognitivist  road  towards  understanding 
how  we  understand.  And  what  we  are  beginning  to  understand 
is  that  reality  is  not  always  just  there,  but  like 
mathematics,  technology  and  religion,  it  is  a  human 
construction . 

The  field  of  Artificial  Intelligence  has  made  similar 
shifts.  A  post-  "artificial  intelligence"  era  confronts 
reality  and  non-reality  with  "virtual  reality."  But  virtual 
reality  is  not  merely  a  science  fiction  concept  in  which  one 
seems  to  step  inside  one's  computer  world.  A  virtual 
library  is  one  in  which  all  the  books  of  the  world  are 
available  to  you,  via  simple  technologies  of  data-base 
searching,  electronic  mail,  and  interlibrary  loan  services. 
A  virtual  classroom  is  one  which  is  tied  in  to  the  entire 
world  through  electronic  technologies. 

All  of  the  above  are  examples  of  the  same  "megatrend". 
Concent rat  ion  on  the  "technical"  is  not  enough.  That 
provides  a  one-sided  view  of  reality.  Even  culture,  in  a 
shrinking  global  village  is  becoming  an  ironic  post-cultural 
phenomenon.     In  short,  we  need  to  be  post-cultural;     we  need 
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to  be  post-cognitive;  we  need  to  be  post-industrial;  we 
need  to  be  poststruc tural ;  we  need  to  be  post-aesthetic. 
And  the  word  which  captures  all  of  that  is  "postmodern." 


Bibliographic  Comment 

The  following  entries  represent  some  of  the  many  writings 
which  suggest  a  move  away  from  the  concept  of  modernity: 

Saul,  J.  R.  (1992).  Voltaire's  Bastards:  The  Dictatorship 
of  Reason  in  the  West .  Saul  pinpoints  Voltaire  as  the 
beginnings  of  the  rational  order,  the  age  of  reason.  But, 
Voltaire's  followers  bastardized  his  ideas  a^.d  his  ideals. 
Reason  and  rationalism  don't  work  when  led  by  bureaucrats 
and  technocrats .  And  that  is  prec  i sely  what  modern  soc  iety 
has  become.  As  one  reviewer  has  put  it,  "John  Ralston  Saul 
wants  to  persuade  us  that  real  enlightenment  lies- not  in  the 
modern  cult  of  Answers,  but  in  the  stubborn,  sceptical  and 
humane  pursuit  of  Questions." 

Barrett,  William  (1986).  Death  of  the  Soul :  From  Descartes 
to  the  Computer .  Barrett  identifies  the  seventeenth  century 
as  the  beginning  of  modernism  and  in  particular,  modern 
science.  While  the  early  philosopher  was  equally  at  home  in 
matters  of  science  and  soul,  contemporary  philosophers  have 
become  pre-occupied  with  data  and  information.  What  has 
been  lost  is  a  philosophy  of  mind,  of  morals,  of  the  soul. 
If  Nietzhche  has  argued  that  "God  is  dead",  Barrett  shows 
that  it  is  us,  that   is  our  soul,  that   is  dead. 

Nisbet,  R.  (1980).  History  of  the  Idea  of  Progress.  Writes 
Nisbet:  "Faith  in  the  dogma  of  progress  is  waning  rapidly  in 
all  levels  and  spheres  in  this  final  part  of  the  twentieth 
century ... [The  reason  is  the]  erosion  of  all  the  fundamental 
intellectual  and  spiritual  premises  upon  which  the  idea  of 
progress  has  rested  throughout  its  long  history."   (p.  9) 
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l-oumlaiions  and  Technology  in  Education: 
A  new  area  of  study  within  the  AECT? 

Al  Januszewski  &  Elisa  J.  Slee 
Syracuse  University 


Our  |n.r,K)se  in  writing  this  paper  is  threefold:  first,  we  would  like  to  descr.be  wha t  it  is  ha 
we  think  that  those  who  affiliate  with  this  "alternatives  group  are  in eresud  in  doing 
second,  we  would  like  to  shew  that  a  large  number  of  these  individuals  study  in  an  area  of 
education  that  is  commonly  known  as  educational  foundations;  third,  we  would I  like  to 
outline  some  of  the  ideas  and  considerations  that  would  be  involved  m  the  creation  ot 
"Division  of  Foundations"  within  the  AECT. 

Admittedly  we  are  relative  newcomers  to  the  AECT.  We  are  somewhat  unclear  as  to  what 
all  this  talk  of  'alternatives'  is  about.  It  was  a  label  that  seemed  to  be  in  place  when  we 
arrived,  and  so,  perhaps  a  little  grudgingly,  we  will  use  it. 

What  are  the  individuals  in  the  alternatives  group  interested  in  doing?  The  term  alternatives 
uroup  is  perhaps  a  misnomer.  After  all,  the  word  alternatives  implies  a  certain  degree  ot 
Sfvity  that  is,  it  is  an  alternative  to  something.  The  AECT,  "V^^wKr  ime 
field  of  educational  technology,  has,  for  some  time,  tried  to  define  he  l.eld  with/for  some 
sort  of  clarity.  The  difficulty  that  the  AECT  has  had  in  successfully  completing ;  this  task 
(defining  the  field  of  educational  technology)  makes  it  equally  difficult  to  define  an 
alternative  to  it. 

You  can  however,  try  to  describe  such  alternatives  in  terms  of  function.  That  is  you  can 
show  a  little  about  how  things  differ  from  the  mainstream  of  current  study  being  done  in 
the  field  of  educational  technology.  Broadly  speaking,  those  that  affiliate  with  the 
alternatives  group  seem  to  differ  from  mainstream  educational  technology  research  wnen 
they  conduct  their  studies  by  at  least  one  of  the  following  forms: 

•  they  use  different  theoretical  bases 

•  they  use  different  research  methodologies 

•  they  ask  different  sorts  of  questions 

I  ike  others  that  study  in  the  field  of  educational  technology,  the  folks  in  the  alternatives 
U roup  both  pose,  and  try  to  answer,  questions  of  "how  to?"  One  of  the  ways  in  which  the 
members  of  the  alternatives  group  can  differ  from  the  "traditional  '  investigators  of 
educational  technology  is  that  they  may  start  with  a  different  theoretical  base  Here  s  an 
example-  How  would  we  design  instruction  without  task  or  hierarchical  analysis.'  mis 
question  would  emanate  from  a  different  theory  base  because  it  would  challenge  some  of 
the  basic  practices  of  our  field,  namely  that  task  or  hierarchical  analysis  is  even  appropriate. 

Another  way  in  which  the  members  of  the  alternatives  group  may  differ  is  in  the  research 
methodologies  employed  to  collect  information.  For  example,  members  ol  this  group  are 
particularly  interested  in  exploring  conceptual  and  qualitative  research  methods  (anothei 
concept  thai  seems  to  defv  consensus  meaning)  in  their  investigative  efforts. 
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The  questions  that  are  posed  by  members  of  this  alternatives  group  can  also  differ 
significantly  from  those  that  are  asked  by  AECT  members  that  are  studying  educational 
technology.  One  example  revolves  around  the  seemingly  very  simple  question,  "Why 
should  we  do  something?1'  (Such  as  recommend  a  particular  instructional  intervention.) 
Often,  traditional  educational  technologists  might  recommend  a  particular  practice  and  then 
study  the  idea  of  this  particular  intervention  to  see  if  it  "works".  Conversely,  many  of 
those  involved  in  the  alternatives  group  would  study  the  action  or  intervention  to  see  if  it  is 
''right'*  or  "good."  In  this  case  a  conscious  effort  would  be  made  to  differentiate  between 
an  idea  that  "works"  and  an  idea  that  is  "right  or  good."  The  thought  is  that  the  two  ideas 
are  simply  not,  a  priori,  the  same.  An  idea  that  "works"  in  a  given  situation  is  not,  de 
facto,  "right"  or  "good"  for  it.  The  emphasis  for  this  sort  of  question  is  less  on  the 
^technical*  (How  can  we  do  it?)  and  more  on  the  'moral'  (Should  we  do  it?)  and  the 
'political'  (What  would  it  mean  for  our  greater  good?). 

Another  distinction  that  might  be  made,  by  virtue  of  having  reconstructed  the  "why" 
question,  is  that  traditional  educational  technology  research  seeks  to  answer  questions  and 
solve  problems.  The  alternatives  group  seems  to  be  interested  in  posing  questions  and 
finding  problems. 

Specifically,  the  areas  of  research  of  interest  to  the  members  of  the  alternatives  group  seem 
to  lie  in  the  arenas  of  philosophy,  qualitative  methods,  history,  sociology  and  cultural 
studies,  in  addition  to  the  more  traditional  areas  of  interest  such  as  psychology, 
communications,  and  teacher  education.  It  is  important  to  note  that  most  of  these  areas  of 
research  interest  are,  more  often  than  not,  considered  as  part  of  programs  that  are  called 
cultural  foundations  of  education",  or  "social  foundations  of  education",  or  simply, 
"foundations  of  education." 

It  is  not  the  case  that  all  of  the  work  that  is  done  by  those  that  affiliate  with  the  so  called 
alternatives  group"  within  the  AECT  falls  into  the  category  that  is  referred  to  as 

educational  foundations.  Neither  is  it  the  case  that  all  of  the  work  that  is  done  in  the  area  of 

educational  foundations  at  the  AECT  is  done  by  those  that  affiliate  with  the  so  called 
alternatives  group".  There  is,  however,  an  area  that  is  common  to  both  thai  is  undeniable 

and  unmistakable. 

Within  the  realm  of  educational  technology,  foundations  seems  to  have  taken  (or  been 
given)  a  back  seat.  Instead,  our  field  has  focused  upon  systems  and  technology  with  an 
increasing  emphasis  on  micro  technologies  (hypercard,  etc.).  So,  the  alternatives  group 
might  be  said  to  be  a  group  of  individuals  interested  in  some  of  the  broader  issues  with 
which  educational  technology  can  be  concerned.  For  example,  educational  technology  and 
us  relationship  to  curriculum  construction  or  its  impact  on  society  and  culture  in  general. 

So,  why  form  a  separate  division?  One  answer  is  that  we  do  this  sort  of  work  anyway.  At 
present,  the  members  that  might  comprise  such  a  division  are  scattered  amongst  the  other 
nine  divisions,  often  a  unheard  voice  in  each.  Often,  unless  by  some  chance  meeting, 
perhaps  at  one  of  the  AECT  receptions,  the  opportunities  to  exchange  common  ideas  and 
perceptions  on  these  sorts  of  issues  are  rare.  In  fact,  the  opportunities  for  intellectual 
exchange  can  be  so  rare  that  we  may  lose  many  of  these  individuals  to  other  more 
traditional  associations  such  as  American  Educational  Research  Association.  There  the 
individuals  in  AFXTs  "alternatives  group"  may  find  themselves  more  comfortable  in  an 
organization  which  has  a  large  membership  studying  similar  things. 
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Why  should  we  run  the  risk  of  losing  these  individuals  to  the  larger  organizations?  Can't 
their  contributions  influence  the  AECT  and  help  us  to  consider  broader  issues  that  will  only 
improve  our  pursuits  as  a  field?  AECT  has  taken  a  step  in  this  direction.  It  established, 
several  years  ago,  the  Annual  Open  Forum  on  the  Foundational  Issues  of  the  Field.  .This  is 
an  annual  session  at  the  national  AECT  conference.  While  this  forum  provides  an  arena  for 
some  exchange,  it  needs  to  be  supplemented  by  additional  sessions  to  increase  the 
opportunity  for  discussion  on  related  topics. 

Finally,  a  study  of  history  will  show  that  some  of  the  best  advances  in  a  field  arise  when  its 
membership  takes  the  time  to  reflect  upon  and  evaluate  the  field's  growth.  A  "Division  of 
Foundations,"  while  engaging  in  ethical  review  and  the  recording  of  history,  might  also 
undertake  this  effort,  on  i\  regular  basis.  The  results  of  this  self-evaluative  effort  could 
inform  the  membership  in  all  divisions  of  the  AECT. 

With  what  would  a  "Division  of  Foundations"  within  the  field  of  educational  technology  be 
concerned?  Some  of  the  topics  would  include:  the  field  and  its  history,  social  concerns, 
and  research.  Issues  that  are  central  to  the  discussion  of  the  field  are  personnel 
certification,  competencies  and  roles;  legal  and  ethical  issues;  and  the  status,  definition,  and 
future  of  the  field.  For  example,  it  would  be  useful  to  examine  the  status  and  future  of  the 
field,  as  well  as  its  relationship  to  other  fields  such  as  psychology. 

Issues  that  are  central  to  the  discussion  of  social  concerns  include  the  relationship  of 
technology  to  education,  the  impact  of  technology  on  culture  and  issues  of  equity  involved 
in  the  use  of  educational  technology.  An  example  of  such  a  social  concern  would  be  the 
unintended  effect  of  computer  based  instruction  on  social  interaction. 

Issues  that  are  central  to  the  discussion  of  research  include  the  purpose  of  the  research,  the 
methodologies  that  are  employed  and  the  setting  of  the  research  study.  Those  in  the 
"Division  of  Foundations"  that  were  interested  in  research  might  reflect  upon  the  type  of 
research  we  conduct  as  a  field.  How  does  it  differ  from  that  conducted  in  psychology? 
communications?  Should  it  differ?  Are  we  making  progress  in  our  research  as  a  field? 
Because  of  the  practical  nature  of  educational  technology,  should  all  of  our  research 
necessarily  be  conducted  in  appHed  settings?  What  research  methods  should  be  employed? 
The  purpose  of  a  creation  of  a  "Division  of  Foundations"  is  not  meant  to  centralize  and 
control  the  discussion  of  each  of  the  above  activities.  Rather,  we  acknowledge  that  the 
potential  member  of  such  a  division  might  also  be  active  in  other  divisions  of  the  AECT.  It 
is  our  opinion,  however,  that  the  proposed  division  would  provide  a  means  for  linking 
individuals  interested  in  pursuing  similar  lines  of  investigation.  And,  exploring  these 
topics  as  an  organized  membership,  individuals  would  be  able  to  inform  members  in  other 
divisions  of  their  findings  which  would  undoubtedly  guide  future  research  and 
development  in  all  divisions. 

This  was  published  in  the  RTI)  Newsletter,  Winter,  1991. 

Please  direct  any  comments  to. 
Al  Januszewski 

Instructional  Design,  Development  &  Evaluation 
Syracuse  University 
330  Huntington  Hall 
Syracuse,  NY  13244-2340 
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The?  Meed  for   an  Unquiet  Pedagogy 


A  pedagogy   is  that  much  more  critical   and  radi-.-U 
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t  e  a  c  h  i  n  g  :i.  I"  s  e  If  is  ^  m  o  r  a  1  a  c.  t  i.  v  i  t  y b  e  c  a  li  s  e  t  e  a  c  h  e  r  s  \  n  t  e  i "  \  e  n  e 
i.  n  t  h e  1  i  v e s  of  t h e  i  r  s t  u d e  n t  s  - 

T  o  v  i  e  w  s  c  h  o  o  1  i  n  g  w  i  t  h  i  r  i  a  p  o  .1  1 t  i  c  a  1  /  e  1 h  i  c  a  i   c  a  n  t  e  >■  t   s  u  g  i: j  e  ■>  t     t !-  i  e- 

need   for   research  and  analyst'/:  that   would   include  but   no l  be 

1  j.  m  i  t.  e  d  t  o ,    a  n  &  x  a  m  i  n  a  1 1  o  n  o  F  t.  h  e   f :*  o  1 1  o  w  i  n  g  5   t.  h  e  n  o  n  n  e  l  i  1  r  a  1  i  l  y 

of  educational   institutions,   and  hence  the  connectedness  of- 

I  now! edge  and  power;   a  critical   analysis  of  current  meanings  anri 

t  he   "of  f  ic  i  al   di scour se "  of  school  i  ng   wi  t  hi  n  soc  i oc  u  ltur*& .! 

a  g  e  r  1  da  s  ?,    visions  o  f  s o c  i  e t.  y  t.  h  a  t.   a  r  e  o  r  i  e  r_i  t  e  d   i  c  * w  a  1  d  s r ,  < ;  i  a  i 

just  :i  c e   a n d  c o m m u. n i  t y ;    a n d   h a v i  1  "1  g   t.  h e  t o o  I.  s  ■  c u n c:: e p t. u a  .!.    <:  1    m ■ « \> o  1  i 

and  language  of  research  that   allow-   <  or   the  mult  3  pie  and  romp'Jt?,' 

f  arms  o f  school i  ng  that  we  exper  i.  ence. 

The  above  commentary  suggests  that  we  need  + o  be  concerned  with 

the  "foundational n   issues  of  schooling.     .i<y   foundat  i  or.a  i    i:  mean 

those   issues  that  raise  Questions  about   the  -'ery  natu«c  nf 

e  i:l  u  c  a  t  i  a  n  a  I.   e x p  e  r  i  e  n  c  e  s «      In  r  a  is  L  n  g   g 1 .1  o  s  \  i  o  n  s  a  b  o  • '.  t    t  • ' .  e   1  "■  <  :« ! .  u  1  o 

of   tne  schooling  experience,,   we  are  ronfronted  with 

c  ...«nt  ■•'  ad   ct  i  ons  „      The  contriKl""1"  ioi'f^  ar-    :«t        r.<;   not        I.    ":  i 1  !■■•« 

+  he?  schooling  p-:p~-''  iences..    but    1.  n  oi.u'    da  » •  to  - day  sxpis-r  j  u.       .  ■: 

:  «:-c:  .i    I.    j.  1.  . 

C  r:  i  - 1 1-  a  d  i  r  t  \.  n  n  s  /  U  ri  t;  e  r  t.  a  1  r"i  t  j.  ■•?  s  o  f '  Sr::  !  "i  o  o  1  i  r  g 

U"-iy  is   it  t  hi  at  educators   find  themsel^*-?   ^..1  ,  M .  the  h-  c«u !  v  ■ 

i .  -■■  ■  c:  h  r  > r"  1  u  g  1  e t::;  /testing  p  r  f r  e  d  u  1 "  <■  c  /'  ( ::•  1-  g  a  n  1. <j  t  i.  o  1 1  ^  1    models,  + .  i  i  f a 

b e  s t   t  h  a  t   (m  o  d  e  r  n  1  ^  t   1 1"  1  i  n  I  ■  1  n g   h  1  a  ^  t  n  o  f  f  or,      p  t   t  h  e v  a 1 '  •    u  r  1  ■ » b  .1  •  ■   *  t ■ 
1  iv  1  p «i c  t   i:  h e  p o s t  fti o d e  1  ■  n  1 {. ; i    w a  1  1  d  •  j  f  t  h e  1  r     !:  u  d ■  r  i  H  .3 "%      i ' h -  ■  m a  1  ■  ■*■*  i  1  m 
i"r '.nmewor  k   for       pi  ai  ni  ng   ]  ]■<"■:■   in  schools.,    what    lifr<   in  :i: «.  h(...  *•  i  •■ 
•should  be   like  is  a  modermst.  concept   rn-.H  ed   in  pr  t?d  1.  ■::  4..ab  l  I.  :  L 
r.«  ."id  cont  r  o  I  *     Sc  hool   reform   1  «■  seen  as   f  1  nt  -   Luru  <  »g  +  he  =s  y   1. 1  -o, , 
h  rA  r  1 1:  e  i.  n  c  r  e  m  g  ■  n  t  a  1   r  e  a  1  r  a  n  g  e?  r  n  e  n  t   n  f  s  c  J't  00]   pi-  a  cr  t  1.  cr  c-  s   is    ■  e  r  1  ■  "■<  .;■ 
s  u  r:«  s  t  a  n  t  i  v  e  c  h  a  n  g  e ,.     \o  t   1  1  v  1.  n  a   i  n  s  r.  1 1  o  c:j  !    >.  s  n  t  ■  i    p  r  o  g '  a  n  1  ; « t *  1  -.:  • . 
L  i  v  1  ng    1.  n  school    1  s   1 1  \  i  ng    1  n  con  f  1 1 1    f  1 »    "»  t^»l 11  '"i  1  '  in    1  - 

rjoi  ng  ori)  -      l.i'/inq   in       hnnl    l=    1. 1  .  •  nci  »«r.     1  f  •-  +  rit'*'d  inrr  :.u  r-"    •  ■  \ 
"1  ea  L"   '!..fa>  nrng   is  goi  ng  nn.i  „     Wort  mri   '  l.i  r >i 1  ^  hu.-  •.•■■•ni  '•  .  .-+• , 
■  t r ugg  1  1  ng  and  negotiating  mooinnjs  i-n   I  hi:-  rtinU'^ifw  i.cr.  ■  :  ■.,  cari 
.1.  *■  •  c.i »  f.   o  n  t-  u  r  1  f.-  L".  1 1  e  d ,   f-'ipcr  i<-r  1  c  j  n  g  <:«   t'  *>■  e  I  '  ri « j  o  f"    1  •  h  a  r  ■  •» "  ,.      i  ?  »•  -  "1  r 
unsettled  f  er-1  1.  ngs ,   e'ippr  1  ehc  i  ng  feel  inns  of  chaos,   arc         *  01 
l"  I-.  p  hi  u  m ;  n  r.  u  n  d  1 1  ion. 


ERIC 


Not    }' *\\r   t  ©moved  from  this  experience  of   living  in  schools  are 
si.ni  iar   experiences  of  "being   in  the  world11.     We  find  ourselves 
\n  '•■;> f ■/ c  i. a  1  settings  that  are  "supposed  to  make  sense";,   that  are 
~uppo<spd  to  be  predictable;,   that  are  supposed  to  be  controlled.. 
:-.rw  i    !    i  Piurm   is  seen  as  fine  tuning  the  social   system.  We 
i  i  - « *  |  >  r  •  1 1  rrl  i-n  incremental   cnange  with  expressions  like  "but   look  how 

*  .»r  *».•■  have-*  came",      Y^t   living,    like*  schooling,    is  not  program- 
abler.     Itr- :  ng   in  the  wor  ld  is  being   in  conflict.     Being   in  the 

w J.  d   i  *  being  within  a  social   setting  that   is  also  a  contested 
J-   r  r  <■«  i  n  n  f   ma  ny  vo  i  ces  ,t    e  peri  e  nc  es  9    interpret  at  ions.      Wo  r  k  i  ng 
iv.  hrouqh  t  ha  conf  1  i ct  ,    st  rugg  1  i  ng  and  negot  i at  i  ng  meani  ngs  on  t  he:- 
conl:i-'i:i>t cd  \+  r*r rain,   can  also  Jeave  on  unsettled  within  the  sociaJ 
bottimj .,     Feelings  of  chaos  also  occur.     This  too   is  part  of  the 
h  u  r  n  a  n  c  o  n  d  1 1  i  o  n .. 

ui<„.-.  /Wi]    and  experience  post  "modernism,    within  a  context  (school/ 
world'    that,  uses  modernist,   language  and  thinking.     This  is 
unsettling,,      Is  modernism  an   imposition  of  rational,  linear, 
t  •:■(':  hrvi  r#  !   processes  to  control    living/schooling?     Is  post- 
.  n  o  ( i  e  i  n  i.  «•>  m  e  ;  •  p  e  r  i  e  n  c  e  s  a  n  d  f  e  e  L  i  n  g  s  of  chaos.,    conflict  and 
struggle-,    all  of  which  brings  uncertainty  because  modernism  is 

♦  I  Mi-  clo:fi  l  nar  it   i  deo i  ogy "? 

t"i  -or  j  pc  u  i  t  u  r  a  3.  p  e  rsp  ec  t.  i  ve  c  an  br  i  ng  t  hese  expe  r i e  nc  es  of 

school /world  together..     Situating  schooling  within  the  larger 

content   of   living   in  the  world   is  not  a  new  idea.  A 

'»o':  .i  ocu  1  i.ura  1   perspective  sees  no  separation  between  the  world 

and  school.      Pi  soc.  locult  ura  1  perspective  understands  the 

h  i.  stoi- ical   and  acknowledges  the  political   and  economic  realities, 

o  i.  i.  a  I    i  :.;> « u e s  a r e  s c h o o I   :i. s s u e s ,    and  vice  versa.      An  u n q u  i e t 
pedagogy h   a  critical  pedagogy;,   confronts  the  social -school --1  i  f::e- 
Norid  and  acknowledges  the  conflict  and  contested  terrain  as  the 
h u m a 1 1  c o n d i t i on. 

An  unquiet/critical  pedagogy   is   informed  by  and  allows  for  the 
technical,   practical   and  emancipatory  forms  of  knowing  within  the 
J.  e  a  i  n  i  n  g  p  r  o  c  e  s  s .     A  n  u  nquiet  /  critical  p  e  d  a  g  o  g  y  u  n  d  e  r  stands  t  h  e 
n •  •   l  o  n s  c ' f  c  u  1 1  u r  a  I   c  a p  i  t  a  I .,    a c  h  i  e  v  e  me  n  t   ideology  a  n d  so c.  i  a  I 
renrodt'CJtzon,    as  well   as  the  social   construction  of  knowledge 
»l-  nowledge  is  created  leather  than  consummed)  . 

■■■n  1 1 1     »  i  L-t  /  cr  it  l  ca  I   pedagogy  ac  k  now  ledges  student  /t  eacher  voice 

•  Mi*  learning  process,.     This  pedagogy  fosters  and  helps  create 
■  K.'fiiO'"  i  j.tn:    schooling  through  confronting  issues  of  social 

,i ir-.t  i  ce«„    the-  LomiTion  good,    and  the  creation  of  community. 

1 1  ihnnlnnv   f     *  pur  I    of   this  wor  ld  of  the  educational,  community- 
I  hi  ■!  k  [  n\  (a  „  ..  „  „ 
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Where   do  you  stop? 

Conversation  and  support  from  the?  Following  has  been  significant 
and   important : 
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Schools  and  Technology  in  a  Democratic  Society: 
Equity  and  social  justice 


Robert  Muffolctto 
Associate  Professor  of  Education 
University  of  Northern  Iowa 
Cedar  Falls,  Iowa  50614 

Internet:  MUFFOLETTO@UNI.EDU 

(This  working  paper  was  written  for  the  1993  meeting  of  the  Association  for 
Educational  Communications,  Research  And  Theory  Division  forum  on  ethics 
and  technology.     Any  comments  and  discussion  is  welcomed.) 

Questions  needs  to  addressed  concerning  the  role  of  public  education  in 
a  democracy.     As  individuals,  and  as  a  profession,  involved  in  the  research, 
development,  production,  and  dissemination  of  educational  experiences  for 
children  and  adults,  we  need  to  consider  what  we  have  created  and  will  create 
in  light  of  social  justice  and  democratic  principals. 

Our  history  is  full  of  attempts  to  design  and  produce  effective  learning 
environments  (In  using  the  term  "our"  I  am  referring  to  those  of  us  working 
in  educational  technology.     I  also  realize  its  not  our  history  but  "a"  history  that 
has  evolved  out  of  conflicts  and  contridictions  representing  various  interest. 
There  are  many  histories,  many  voices.).     We  have  consumed  various  learning 
theories  and  have  produced  various  formats  for  the  delivery  of  curriculum 
materials.     Our  purpose  has  been  to  increase  the  effectiveness  of  teaching 
materials  and  the  efficiency  of  the  learning  process. 

Our  field  is  grounded  in  logical  positivism,  capitalism,  and  a  19th  and 
20lh  century  notion  of  progress  and  classical  realism.     Technology,  both  as 
machine  and  as  system,  was  and  is  linked  with  modernism  and  progress. 
Reality,  especially  social  reality,  and  the  stories  told  about  it  by  experts,  is 
understood  to  exist  outside  the  individual  and  has  for  the  most  part  gone 
unquestioned  and  unrecognized  by  researchers  in  our  field.     Beneath  all  of 
this  lies  the  ideology  of  the  machine  and  the  expert  (Muffolctto,  in  press). 

Our  field  has  strived  to.  create  through  various  presentational  formats 
"a"  reconstructed  reality.     Most  of  the  debate  in  these  attempts  has  centered  on 
the  veracity  of  the  experience;  docs  it  feel  real,  docs  it  reflect  reality,  is  it 
efficient,  and  is  it  effective  in  its  delivery.    There  has  been  little  debate  on  the 
consequences  of  those  strivings  for  a  reality  on  the  lives  of  real  people  and 
their  culture.     With  the  recent  developments  in  virtual  reality  and  multi-media 
hardware  and  software  we  must  begin  and  continue  our  attempts  to  address  the 
psychological,  social  and  political  implications  and  effects  of  what  "we"  do  as 
perceived  experts,  as  educational  technologists.     No  longer  can  we  afford  to 
claim  the  neutrality  of  a  modernist  tradition  or  the  non-historical 
consciousness  which  accompanies  a  positivist  discourse  towards  reality  and 
experience.     As  educators,  researchers,  and  developers  of  learning 
experiences  we  must  find  avenues  and  entry  points  for  debates  and  practices 
that  argue  and  provide  for  spaces  that  support  and  maintain  democracy  and 
social  justice.    The  first  step  I  believe  is  to  recognize  ourselves  for  what  we  are; 
a  social,  historical,  and  cpistcmological  construction.    The  second  step  is  to 
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define  whal  wc  mean  by  democracy  and  social  justice.     The  third  is  to  position 
our  definitions   in   practice.  fe 

Technology  as  a  medium  for  discourse 

Technology  is  more  than  a  tool,  it  is  a  medium  which  effects  how  wc  think  and 
interact  with  others  and  machines  (  Rhcingold,  1991).     It  is  a  form  which  not 
only  controls  and  limits  discourse  but  determines  the  nature  of  the  content  as 
well  (Postman,  1992).     Technology  is  more  than  access  to  information  and 
learning  experiences.        Technology  determines  the  nature  of  that  information 
as  welt  as  our  understanding  of  it.    As  a  medium  of  experience  (discourse), 
technology  effects  our  consciousness,  our  visions,  and  our  expectations.  The 
"wet ware"  of  a  modernist  technology  constructs  the  individual  as  a  subject 
(Berger  and  Luckmann,  1966;  Muffolctto,  1991).     The  technological  medium  is 
more  than  a  mind  manager  and  a  reality  simulator,  it  is  a  consciousness 
generator   —an   ideological   horizon  line. 


Information 

If  technology  is  to  provide  us  with  access  to  information,  there  arc  a  number 
of  issues  that  must  be  considered  and  addressed.     Simply  providing  access  to 
information  is  not  enough  in  a  social  context  were  historically  access  has  been 
limited  to  the  wealth,  gender,  and  race  of  the  individual  or  community.  Access 
to  information  must  also  include  equity  in  access  to  ways  of  "thinking"  about 
information.     If  information  is  to  be  used  to  empower  people  within  the 
democratic  tradition,  then  educational  experiences  must  provide  a  means  for 
equal  access  to  ways  of  thinking  as  well  as  valuing  different  ways  of  thinking. 

To  have  information  and  not  know  what  to  do  with  it,  is  as  serious 
problem  as  not  having  information  at  all  (Of  course  this  begs  the  question 
about  the  nature  of  information,  cpistcmology,  legitimization).  Individuals 
who  historically  have  been  positioned  on  the  margins  of  power  and  knowledge 
because  of  their  culture,  their  economic  class,  their  gender,  their  race,  or 
their  religion,  may  have  been  given  equal  access  to  information  (even  in 
limited  ways),  but  not  ways  of  knowing  (thinking).     For  example,  lac  cultural 
ways  of  making  sense  in  the  United  Slates  has  been  limited  to  primarily  one 
cultural   and  economic   framework  (white,  middle-class,  male,  and  European). 
How  one  thinks  about  the  world  and  one's  self  in  i:  determines  the  rationale 
for  understanding  why  things  arc  the  way  they  arc  (common  sense),  and  not 
why  reality  is  thought  about  in  that  manner. 

How  one  thinks  about  the  world  as  well  as  sell,  is  how  one  has  been  told 
to  act  and  think  in  relationship  to  self  and  others.     Having  information,  but 
not  divergent  ways  of  thinking,  maintains  the  individual   and  the  community 
in  a  powerless  relationship  to  those  who  do.     Having  access  to  information  may 
create  a  false  consciousness  resulting  in  less  real  power  than  before. 

Simulations    as  experience 

Virtual  reality,  as  a  technology  of  experience,  poses  a  number  of 
questions.     First  and  most  basic,  wc  must  consider  what  the  relationship  is 
between  a  "virtual"  reality  and  something  wc  call  reality.     Is  it  good  enough  to 
be  concerned  with  only  the  veracity  of  the  experience  and  :ts  correspondence 


So 
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to  what  is  believed  to  be  out  there?  (The  physical  and  social  sciences  can  be 
separated  here,  but  questions  concerning  how  we  know  reality  and  truth  are 
essential  to  both  paradigms.)    In  doing  so  we  must  offer  up  for  analysis  the 
manner  in  which  we  came  to  think  about  what  is  out  there.    We  tend  to  forget 
that  our  understanding  of  what  we  think  is  out  there  is  a  result  of  the  tools  we 
use  to  explore  it,  the  language  we  use  to  construct  it,  and  the  context  or  system 
(Goodman.  1978)  used  to  understand  it.    Change  the  tool,  the  language,  or  the 
system,  and  reality  differs.     As  individuals  concerned  with  the  creation  of 
simulations,  other  worlds,  we  can  not  forget  that  we  exist  within  a  social 
reality,  a  virtual  realty  of  sorts.     We  must  also  recognize  that  through  discourse 
management,  constructed   reality  has  become  reified   and  objectified. 

Second,  if  virtual  reality  is  understood  in  terms  of  simulations,  looks, 
feels,  and  sounds  alike,  can  not  that  simulation  be  understood  as  a  discourse. 
As  a  discourse  virtual  reality  must  be  analyzed  as  any  other  discourse? 
Borrowing  from  Chcrryholmcs  (1988)  wc  would  need  to  question  virtual 
reality  by  asking:  Who  is  controlling  the  discourse  (reality)?;  Who  is  allowed  to 
speak  and  listen?;  What  is  being  said?;  Who  benefits  from  what  is  being  said?; 
and  What  is  not  being  spoken  about? 

Any  simulation  or  virtual  reality  must  be  considered  from  two  different 
perspectives.    .On  one  side  wc  must  consider  who  is  constructing  the  world  to  be 
experienced  by  users  (students,  teachers,  workers,  infonauts)  Notions 
concerning   hypertext   environments,   interactive   video,   and    virtual  reality 
include  authors  and   readers,  guides  and  travelers,  navigators  and  explorers. 
No  technological  environment,  as  a  system,  is  authorlcss.     Every  author,  every 
programming  production  team,  every  navigator,  holds  a  world  view,  an 
ideological  perspective,  a  consciousness  about  self  and  others.     On  the  other 
side,  what  are  the  social,  psychological,  and  political  effects  of  a  constructed 
world  on  the  readers  of  the  virtual  text. 

 Learning 

How  "we"  come  to  be  as  subjects,  as  social  beings,  is  a  result  of 
experiencing  constructed  texts  (texts  is  used  here  in  a  post-modernist  manner) 
and  meanings  (Bclscy,  1980).     All  texts  arc  hegemonic  and  are  part  of  a  larger 
discourse  encoded  with  meanings,  values,  and  ideological  perspectives  on 
others  and  self.    How  and  what  wc  learn  about  a  social  world  is  the  result  of 
experiences  with  various  discourses  about  that  world.     In  doing  so,  we  either 
reproduce  dominate  meanings  and  ways  of  knowing  or  offer  oppositional  and 
alternative  discourses  (Hall.  Hobson  &  Willis,  1980).     In  cither  case,  individuals 
as  members  of  interpretive  communities  (Fish,  1980)  understand  a  reality  to  be 
as  it  is,  to  be  real  and  truthful,  because  of  their  experiences  with  various 
formative  and  informative  discourses  (Ellsworlh&  Whatlcy,   1990).  Questions 
referring  to  equity  and  social  justice  emerge  out  of  a  discourse  on  social 
learning,  power  and  control,  benefit,  and  history. 

School  Reform  and  Technology:  Towards  Social  Inquiry  and  Justice 

Curriculum   materials,   delivery   systems,   and   learning   environments   may  be 
understood   as  social   texts,   representational   in   nature,   always  overtly  referring 
to  something  else,  while  covertly  referring  to  themselves  as  a  formative 
medium.     The  form  and  content  of  learning  environments  not  only  speak  to 
methods  and  content,  but  also  refers  to  ways  of  thinking  and  knowing. 
Thinking   about   all   learning  environments,  methodologies,   and   contents  as 
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representational,  as  ideological  representations,  adds  another  dimension  to  our 
thinking   about   schooling,  technology,   and  change. 

Change  always  refers  to  difference.     In  education  as  well  as  business 
change  is  considered  as  a  reply  to  some  identified  problem.     How  these 
problems  are  identified  is  as  important  to  understand  as  what  the  problem  is 
reported  as  being.     Needs  assessments,  goal  development,  and  vision  statements 
refer  to  a  history,  the  present,  and  to  a  fulurc.     Futures  arc  normally  related  to 
notions   of  progress. 

What  the  problem  is,  is  determined  by  who  (who  being  not  an  individual  but  a 
community)  is  asking.     If  problems  and  solutions  arc  defined  in  terms  of 
efficiency,  outcomes,  and  management,  the  problems  and  solutions  will  be  of 
one  nature.     If  problems  arc  contcxtuali/xd  in  a  discourse  of  democracy  and 
social  justice,  efficiency,  outcomes,  and  management  may  be  part  of  the 
solution  but  to  "what  and  how11  they  refer  to  will  be  different.     As  education  in 
the  United  States  considers  why  and  how  it  must  change,  technology  as  a 
medium   which   effects  knowing,  institutional   and   individual   relationships,  as 
well  as  a  sense  of  self  and  others,  must  be  better  understood  within  a  discourse 
of  democratic  ideals.     The  problem  needs  to  be  redefined.     (Again,  the  language 
has  to  be  problemized  whenwe  consider  that  there  is  not  one  education,  but 
many.) 

Critical   Theory   and   Educational  Technology 

Critical  theory  offers  an  entry  point  lor  unpacking  the  values, 
assumptions,  and  practices  of  educational  technology.     From  a  post-modernist 
perspective  critical  theory  ciaims  no  absolute  authorship.     It  declares  its  own 
subjectivity  and  ideological  construction.     As  a  theory  working  within  a  post- 
modernist  tradition,  those   who  practice  critical   theory   arc  concerned  with 
qucstirns  of  power,  control,  and  cpistcinology  as  social  constructions  with 
benefits  to  some  and  not  to  others. 

A  critical  theory  of  educational  technology  would  be  concerned  with 
issues  of  consciousness  and  cpistcmology,  power  and  control,  institutional  and 
individual   relationships  (Fccnbcrg,   1991).     Questions  concerning  equity  and 
social  justice,  and  the  construction  of  individuals  as  subjects  within  an 
ideological  discourse  would  be  critical  to  the  unpacking  and  redefinition  of  the 
theories  and  practices  of  educational  technology.     A  major  impact  of  critical 
theory  on  the  field  of  educational  technology  would  be  to  recognize  itself  as  a 
social  construction  with  a  history  of  conflicts,  struggles,  and  contradictions. 
In  understanding  the  social  and  historical  nature  of  the  field,  the  values  and 
assumptions  which  arc  expressed  through  various  discourses  would  be  open 
for  analysis. 

Conclusion 

Schooling  in  rc Heeling  a  democratic  society,  requires  a  society  lo  be 
democratic,  non-racist,  non-sexist,  and  not  class  based.     In  positioning 
education  as  a  major  socializing  institutions,  wilh  a  major  role  in  forming  the 
worldvicws  and  subjectivities  of  its  participants,  the  products  and  processes  of 
educational  technology  do  play  a  major  role  in  how  communities  of  individuals 
think  about  others  and  sell.     A  critical  theory  position,  breaking  from  the 
common  sense  reified  world  offered  by  modernist  and  posilivist  alike,  would 
need  to  address  issues  concerning  the  function  of  schooling  and  a  technology 
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of  instruction  in  a  democratic  society. 
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"...critical  theory  turns  its  face  resolutely  away  from  all  forms 
of  dogmatic  authoritarianism,  asserting  the  necessity  of  a 
transformation  of  conscious  control  over  system  imperatives 
through  democratic  communication  processes"  (Young,  1990,  p.  55). 
Critical  theorists  aim  to  move  people  to  freedom  via  a  changed 
consciousness  brought  on  by  democratic  communication.  Our 
transformed  consciousness  is  to  include  not  only  the  technical  and 
practical  forms  of  knowledge  that  now  dominate  our  lives  but  the 
emancipatory  forms  of  knowledge  which  express  not  only  what  can  be 
done  but  which  should  be  done,  which  put  truthfulness  and 
sincerity  on  a  par  with  being  right,  which  are  subjective  as  well 
as  objective  (Habermas,   1984,   1987;  Ewert,  1991).     Put  similarly, 
instrumental  reason  is  to  be  re-unified  with  political-ethical 
reason  (Young,  1990,  p.  17). 

A  primary  method  of  a  critical  theory  of  educational  technology 
must  be  to  expose  the  ways  ideologies  are  developed  and 
perpetuated  via  language  forms.     For  present  purposes,  ideologies 
are  perpetuated  via  "languages"  which  include  technologies. 

Ideology  can  be  described  as  "The  values  and  interests  of  ruling 
classes  and  elites  [that]  are  installed  in  the  very  design  of 
rational  procedures  and  machines  even  before  these  are  assigned  a 
goal"  (Feenberg,  1991,  p.  14)  as  well  as  the  values  and  interests 
in  the  goals  of  technology.    A  television  expresses  the  political- 
ethical  interests  of  the  Sony  Corp.  to  have  power  over  consumers, 
especially  via  money  and  technology,  and  especially  in  such  a  way 
as  to  keep  the  technical,  practical,  and  political-ethical 
knowledge  held  by  the  corporation  away  from  consumers  and  others. 

Language  can  be  taken  primarily  to  be  spoken  or  written  Spanish  or 
Russian,  for  instance,  but  it  also  is  a  more  technical  form  such 
as  computer  programming  languages,  it  is  representational  visual 
images  such  as  paintings  or  holograms,  and  it  is  even  symbols  such 
as  computers,  cars,  and  beer  bottles  and  the  ways  they  speak  to 
humans • 

Technology  can  be  the  belief  in  and  application  of 
rational /systematic  thinking.     It  is  manifested  in  a  belief  in 
technology,  in  products  such  as  gas  chambers,  and  in  processes 
such  as  science.  * 

Educational  technology  can  be  any  technology  and  a  belief  in  any 
technology  used  to  change  humans,  though  change  usually  is 
prescribed  and  occurs  within  formal  bounds  of  schooling. 

More  than  exposing  ideologies  in  languages,  the  method  of  critical 
theory  (and  so  of  a  critical  theory  of  educational  technology) 


includes  "immanent  critique,  which  proceeds  through  forcing 
existing  views  to  their  systematic  conclusions,  bringing  them  face 
to  face  with  their  incompleteness  and  contradictions,  and, 
ultimately,  with  the  social  conditions  of  their  existence"  (Young, 
p.  18), 

For  example,  educational  technology  perpetuates  students'  lack  of 
critical  thinking  because  it  promotess  mostly  technical  interests 
and  rational-instrumental  thinking.     It  fosters  the  dominance  of 
financiers,  politicians,  militarists  and  professional  educators 
over  students.    Further  then,  educational  technology  limits 
democracy  and  sincere  communication  because  people  are  less  able 
to  think  critically  about  politics  (Koetting,  1983;  Nichols, 
1990) . 

A  technical  and  practical  ideology  rather  than  a  democratic- 
communicative  ideology  appears  predominant  in  all  of  education. 
That  is,  studants  and  teachers  are  not  responsible  for  knowledge 
and  education  but  for  fulfilling  the  desires  of  others,  especially 
the  desires  to  have  power  and  make  money.    As  a  result  of  this 
dominance,  kept  from  authentic  motivation,  students  don't  want  to 
know  much  and  can't  know  much,  teachers  don't  learn  or  teach  much, 
and  people  and  society  loose  hope  and  act  non-communicatively ,  on 
the  whole.     Schools *are  filled  with  demoralized  people  who  don't 
know  or  often  care  that  they  are  not  communicative.     30-40  per 
cent  of  American  students  drop  out  of  school. 

If  we  are  serious  about  the  "education"  in  "educational 
technologist,"  we  must  critically  study  this  dominance.  Such 
study  is  ethical/moral  because  of  its  potential  to  encourage 
greater  fulfillment  of  human  communication.     Freedom  of 
communication  is  moral.    We  will  make  human  and  ecological 
progress  to  the  extent  that  we  re-acquaint  technical  reason  with 
moral  reason* 

Carr  and  Kemmis  (1986)  offer  examples  of  ways  to  critically  study 
educational  technology,  as  do  critical  pedagogists  such  as  Weiler 
and  Mitchell  (1992),  as  do  Randy  Koetting,  Rhonda  Robinson,  and 
the  other  presenters  at  this  symposium. 

Problems  with  a  critical  theory  of  educational  technology  include: 

1 .  Is  social-material  progress  as  meaningful  as  critical 
theorists  indicate/hope? 

2.  Will  a  higher  rationality  will  be  more  helpful  to  human 
dignity,  if  a  lower  one  has  gotten  us  into  the  scrapes  we're  in 
now? 

3.  Can  critical  theorists  reach  their  intended  audience 
(proletariate/teachers/students)  with  the  technical  language  they 
employ  (Young,  1990) 

Maybe  because  of  these  problems,  critical  theory,  at  least  in  its 
academic  forms,  has  not  reached  American  students  or  school 
teachers  in  any  widespread  way.    Virtually  no  students  have  heard 
of  "critical  theory."    Beyond  having  heard  the  phrase  "critical 
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theory,"  and  more  importantly,  most  students  are  not  dteply 
sensible  that  their  communicative  (so  political-ethical)  freedoms 
are  submerged  and  restricted.    They  can't  be  in  today's 
predominant  modes  of  education. 

The  most  crucial  question  for  educational  technologists  remains: 
How  can  we  get  students  to  participate  in  all  aspects  of 
educational  technology  when  the  likes  of  freedom,  justice, 
equality,  and  physical  existence  are  hardly  open  to  conversation? 
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The  authority  of  the  text  has  gone, 
probably  stolen.  The  literature  police  called  in 
the  philosophy  detectives.  Together,  they 
uncovered  proof  that  Jacques  Derrida  had  his 
eye  on  it  for  a  while.  Also,  around  the  same 
time  it  was  noticed  missing,  he  began  traveling 
widely,  popping  up  at  many  seminars  and 
lectures  in  Europe  and  North  America. 

Interrogations  of  possible  witnesses  show 
that  the  meaning  of  meaning  is  shifting.  As 
soon  as  a  lead  is  established,  contradictory 
information  drives  the  investigation  onto 
another  track.  Anyway,  whether  he  did  it  or 
not,  the  plan  for  the  theft  seems  to  appear  in 
Derrida's  Structure,  Sign  and  Play  in  the 
Discourse  of  the  Hutnan  Sciences  which 
identifies  paradoxes  in  structuralist  reading. 
*  That  conference  paper  reveals  the  pretense  of 
contextual  stability  and  introduces  a  new  term: 
deconstruct! on.  Not  long  after,  Derrida 
provided  another  clue  by  Xing- out  words  as  a 
visual  sign  that  signs  themselves  are  unstable 
and  changed  as  soon  as  they  are  understood. 

Other  suspects,  Roland  Barthes  and  Michel 
Foucault,  died  under  unusual  circumstances 
and  left  behind  suspicious  documents.  Between 
them,  Derrida,  Barthes  and  Foucault  have  over 
60  English  titles  listed  in  Books  in  Print  but 
there  seems  no  indication  of  conspiracy.  These 
three  French  writers  considerably  influence  the 
contemporary  teaching  of  English,  language, 
literature,  social  studies,  and  similar  subjects  at 
the  secondary  and  college  levels.  As  a 
consequence,  students  may  be  becoming  less 
intellectually  docile  as  classroom  learners  and 
less  malleable,  after  graduation,  when  exposed 
to  new  employee  training. 

Would-be  followers  surface  from  time  to 
time,  apparently  expecting  a  rendezvous,  but 
Derrida  disavows  them-just  as  he  did  Socrates 
who  was  picked  up  at  the  scene  of  the  crime. 
Clearly  the  wrong  guy  but  surely  up  to  no 
good,  Socrates  was  arrCvStcd  on  the  related 
charge  of  teaching  eflcctivcly-which  shows 
the  danger  of  educational  communications  and 
technology. 


Derrida  challenges  the  assumption  that 
spoken  language,  the  most  immediate  of 
communication  media,  can  be  accepted  as  the 
closest  representation  of  thought.  It     a  non- 
neutral  medium  shaped  by  ideology  and  bias. 
No  way  of  communicating,  whether  in  speech 
or  writing  or  painting,  for  example,  is  more  or 
less  direct  or  unequivocally  better.  Language 
has  become  endlessly  self  referential  and  this 
belief  is  unlikely  to  be  reversed  even  if  the 
authority  of  the  text  is  restored.  The  tradition 
of  searching  for  an  author's  intended  meaning 
in  a  text  is  reduced  to  a  mere  preconception. 

Other  than  a  bungled  arrest  and  deportation 
from  Prague  (it  was  obvious  the  drugs  had 
been  planted)  Derrida  has  a  clean  record.  On 
the  face  of  it,  he  is  a  law  abiding  citizen  and 
even  writes  with  a  word  processor 
(Macintosh).  Nevertheless,  he  once  expressed 
misgivings  about  reading  aloud  an  oath  to 
respect  the  rules  of  a  library.  Politically,  he  has 
been  active  with  the  communal  International 
College  of  Philosophy,  administered  by  Jean- 
Francois  Lyotard.  This  project  developed  out 
of  a  collective  appeal  to  release  philosophy 
from  the  prison  of  higher  education.  It  may 
only  be  coincidence  that  the  Groupe  de 
Recherches  sur  TEnscignmcnt  Philosophiquc 
is  known  by  its  initials:  GREPH.  The 
concealed  use  of  the  Unix  command  "grcp"  is 
possibly  a  nise  to  cover  a  deep-rooted  interest 
in  telecommunications.  In  the  United  States, 
for  example,  J.  David  Bolter,  Michael  Joyce 
and  George  P.  Landow  have  gained  attention 
for  applying  Derrida's  ideas  to  designing  and 
understanding  the  design  of  computerized  texts 
and  education. 

There  are  social  consequences  here  that 
threaten  to  disrupt  power  and  established 
authority.  How  can  minds  converge  if  it  is  a 
fallacy  to  think  of  thought  as  language  because 
language  itself  is  undccidablc?  However,  at 
this  point,  the  evidence  against  Derrida,  the 
likely  ringleader,  still  remains  only  hearsay 
and  it  is  now  becoming  increasingly  uncertain 
exactly  what  has  disappeared. 


77 


Title: 

Curiosity  as  a  Personality  Variable  Influencing  Learning 
in  an  Interactive  Environment 


Authors: 


Marilyn  P.  Arnone 
Barbara  L.  Grabowski 


79  1 


ABSTRACT 


This  study  investigated  the  effect  of  curiosity  in  first  and  second  grade 
children  as  an  individual  difference  variable  in  learning  in  a  computer-based 
interactive  learner  control  environment.  The  instruction  was  an  art  education 
lesson  and  contained  both  facts  and  concepts.  Learner  control  varies  by  the  degree 
of  instructional  control  which  a  student  has  over  his/her  own  learning.  In  a 
"complete  learner  controlled  lesson,"  the  learner  makes  his/her  own  decisions  about 
issues  such  as  content  selection,  sequencing,  pace,  and  remediation.  In  a  "learner 
controlled  with  advisement  lesson,"  the  learner  makes  the  same  type  of  decisions 
but  is  provided  advisement  about  the  decisions  and  other  learner  control  options. 

This  study  used  two  learner  control  treatments:  1)  learner  control  without 
advisement  (No  Advisement)  and  2)  learner  control  with  advisement  (Advisement), 
High  curious  subjects,  who  generally  prefer  a  higher  degree  of  unfamiliarity  and 
uncertainty,  were  predicted  to  perform  better  in  a  learner  control  environment  than 
low  curious  children;  both  treatments  represented  a  learner  control  environment. 
All  children  regardless  of  curiosity  level  were  expected  to  perform  better  in  the 
Advisement  condition  than  in  the  No  Advisement  condition. 

As  predicted,  there  was  a  significant  difference  in  achievement  scores  in 
favor  of  the  high  curious  children  in  both  learner  control  conditions.  The  results  of 
this  study  suggested  that  differences  in  curiosity  influenced  performance  within 
either  type  of  learner  control  CBIV  lesson.  The  differences  were  independent  of 
grade  level  or  gender.  Research  has  shown  curiosity  to  be  independent  of  IQ, 

Although  there  were  no  significant  differences  overall  for  treatment,  there 
was  a  significant  interaction  between  grade  level  alid  treatment.  Grade  one 
subjects  performed  significantly  better  in  the  Advisement  condition  than  in  the  No 
Advisement  condition  while  grade  two  subjects  performed  significantly  better  in  the 
No  Advisement  treatment.  Examination  of  in-treatment  data  indicated  that  the 
grade  one  subjects  heeded  advisement  more  than  the  second  grade  subjects  which 
might  explain  why  that  group  performed  better  in  the  No  Advisement  treatment.  A 
possible  explanation  for  the  unexpectedly  high  scores  of  second  graders  in  the 
Advisement  treatment  may  have  been  due  to  the  predominance  of  high  curious 
subjects  in  that  treatment  group;  high  curious  subjects  would  be  expected  to 
perform  better  in  a  situation  of  greater  uncertainty  than  low  curious  subjects. 

Implications  of  the  findings  for  instructional  designers  and  educators  in 
planning  instruction  are  discussed. 
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INTRODUCTION 


A  RECIPE  FOR  LEARNING 

Take  a  student,  place  him  in  a  situation 
of  moderate  uncertainty  about  some  topic 
and  get  out  of  his  way  while  he  gets 
excited  and  attentive  and  directs  his 
exploration  to  the  source  of  his  uncertainty. 
Moreover,  research  has  demonstrated 
that  he  will  enjoy  his  exploration  and 
the  accumulation  of  knowledge. 

Day,  1982,  "Curiosity  and  the  Interested 
Explorer"    p.  19 

Research  on  individual  differences  in  cognitive  development  has  largely 
centered  on  intellectual  aptitudes  somewhat  diminishing  the  importance  of  other 
aspects  of  intellectual  development,  such  as  intellectual  styles  (Henderson  &  Gold, 
1983).  These  styles  include  cognitive  styles,  creativity,  and  the  subject  of  this  study, 
curiosity.  Stimulating  curiosity  in  the  academic  environment  is  a  challenge  for 
educators.  Curiosity  motivates  scholarship.  A  curious  student  will  want  to  explore 
and  learn.  Curiosity  is  motivation  which  is  intrinsic  as  opposed  to  motivation 
which  comes  as  a  result  of  external  incentives,  such  as  rewards  or  threats  of  test 
taking. 

However,  not  everyone  is  equally  curious.  Day  (1982)  discusses  the 
tendency  for  some  students  to  become  curious  more  readily.  They  enjoy 
opportunities  to  explore  at  will  while  others  are  less  tolerant  of  unfamiliarity  and 
uncertainty  which  are  almost  always  present  in  a  situation  in  which  curiosity  is 
aroused. 

Not  only  do  people  differ  in  curiosity  as  a  personality  trait  but  environment 
is  thought  to  play  a  part  as  well  in  the  development  of  curiosity  in  students.  Some 
students,  according  to  Day  (1982),  may  have  had  their  disposition  to  be  curious 
dampened  at  an  early  age  by,  for  example,  being  punished  for  asking  too  many 
questions  or  exhibiting  curiosity  in  numerous  ways.  It  is  up  to  the  teacher  (or,  in 
the  case  of  technology,  the  teacher  function*)  to  give  these  pupils  special  training 
"so  that  they  may  become  curious  once  again"  (p.  21).  Although  future  studies 
should  address  whether  trait  curiosity  can  actually  be  enhanced,  the  focus  of  the 
present  study  was  to  explore  the  effect  existing  individual  differences  in  curiosity 
level  have  on  learning  in  situations  which  differ  in  amounts  of  learner  control. 
Since  learner  control  situations  involve  different  amounts  of  uncertainty,  awareness 
of  individual  differences  in  curiosity  potential  is  essential  to  the  planning  and 
design  of  instruction. 

Planning  for  how  to  deal  with  curiosity  is  not  only  a  challenge  for  educators 
but  also  for  instructional  designers  and  developers  of  the  many  interactive  learning 
technologies  that  are  emerging.  The  present  study  utilized  computer-based 
interactive  video  (CBIV).  CBIV  involves  the  interfacing  of  computer  technology  and 
video  in  such  a  way  as  to  allow  active  user  participation  and  the  opportunity  to 
modify  pace,  content  sequencing,  remediation  choices,  etc.,  as  a  result  of  a  viewer's 
response.  The  amount  of  interactivity  in  a  CBIV  lesson,  however,  is  based  to  a 
large  extent  on  the  degree  of  learner  control  which  is  afforded  by  the  lesson. 
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A  number  of  studies  exist  on  issues  related  to  learner  control  (e.g.,  Balson, 
Manning,  Eber  &  Brooks,  1985;  Gay,  1986;  Hannafin  &  Colamaio,  1987;  Gay, 
Trumbell  and  Smith,  1988)  although  there  are  relatively  few  studies  which  explore 
this  area  with  young  learners.  Prior  research  suggests  that  higher  achievement 
scores  are  possible  when  there  is  a  component  of  advisement  included  in  a  learner 
control  lesson  (Tennyson,  1980;  Johansen  &  Tennyson,  1983;  Tennyson,  1984;  etc  ). 
This  finding  was  found  even  for  children  (Arnone  &  Grabowski,  1992).  There  is 
also  substantial  research  in  the  area  of  curiosity  and  motivation  (e.g.,  Beriyne, 
1954;  Beriyne,  1960;  Maw  &  Maw  1964;  Maw  &  Maw,  1965;  Maw  &  Maw,  1966; 
Day,  1968;  Wilson,  Stuckey  &  Langevin,  1972;  Vidler  &  Rawan,  1975).  These  areas 
of  research  exist  independently  of  each  other,  and  as  such,  little  is  understood 
about  the  effect  of  personality  variables  such  as  trait  curiosity  on  learning  in  a 
learner  control  environment  or  about  the  interaction  of  such  variables  within 
different  learner  control  situations.  Questions  remain  regarding  the  effect  on 
performance  of  one's  predisposition  to  be  curious.  Will  the  initially  more  curious 
child  be  more  capable  of  accepting  greater  degrees  of  control  over  their  learning 
because  their  optimal  arousal  level  is  higher  (and  therefore  their  comfort  zone  for 
uncertainty)?  The  purpose  of  this  study  was  to  investigate  the  effect  of  curiosity  as 
a  personality  variable  on  learning  in  an  interactive  learner  controlled  environment. 

CURIOSITY 

The  Construct 

Curiosity  is  often  associated  with  exploratory  behavior.  The  purpose  of 
exploratory  behavior  is  to  keep  one's  level  of  stimulation  at  what  Beriyne  (1960)  in 
his  neurophysiological  model  of  arousal  calls  the  "tonus  level."  (The  tonus  level  is  a 
moderate  pleasurable  level  of  stimulation  at  which  an  individual  functions  most 
effectively.)  Beriyne  identified  two  forms  of  exploratory  behavior,  "diversive 
exploration"  and  "specific  exploration."  Diversive  exploration  occurs  as  a  person 
seeks  new  experiences  or  relief  from  boredom  whereas  specific  exploration  is 
encountered  in  situations  in  which  there  is  conceptual  conflict  arising  out  of 
uncertainty  or  novelty.  Specific  exploration  is  described  within  the  context  of 
epistemic  curiosity,  that  is,  "the  brand  of  arousal  that  motivates  the  quest  for 
knowledge  and  is  relieved  when  knowledge  is  procured"  (Beriyne,  1960,  p.  274). 
Epistemic  curiosity,  then,  motivates  exploration  which  resolves  the  conceptual 
conflict  and  returns  the  individual  to  a  pleasurable  tonus  level.  It  is  important  to 
note  that  small  increases  in  stimulation  are  considered  pleasurable  because  of  the 
resultant  return  to  the  tonus  level  when  the  conflict  is  resolved.  However,  large 
increases  are  not  considered  pleasurable  and  will  result  in  the  individual  making 
attempts  to  return  to  the  tonus  level.  One  way  the  individual  may  attempt  to 
return  to  the  tonus  level  might  be  to  reduce  the  stimulation  by  actuslly 
withdrawing  from  it,  for  example. 

Berlyne's  theory  of  curiosity  involves  a  construct  known  as  "collative 
variability."  Collative  variability  relates  to  how  complex  a  stimulus  is  and  how 
easy  or  difficult  it  is  for  the  individual  to  collate,  that  is,  to  compare  the  stimuli  to 
that  which  the  individual  is  familiar.  Collative  properties  have  the  potential  to 
increase  arousal  level  and  induce  curiosity.  Day  and  Beriyne  (1971)  note,  however, 
that  individuals  differ  in  their  tolerance  and  preference  for  arousal  potential 
While  some  individuals  may  react  with  interest  and  exploration  to  a  given  amount 
of  arousal  potential,  another  individual  may  become  inefficient  and  withdraw  from 
the  same  stimulus. 
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Day  (1982)  describes  this  arousal  theory  as  a  curvilinear  relationship 
between  efficiency  (in  terms  of  learning)  and  arousal.  The  optimal  level  of  arousal 
is  one  in  which  the  individual  enters  a  Zcne  Of  Curiosity  and  is  willing  to  apprcach 
or  explore  his/her  environment  in  order  to  resolve  the  cognitive  conflict  that  was 
initiated  by  the  stimuli.  When  the  conflict  is  resolved  generally  by  seeking 
answers,  etc.,  the  individual  returns  to  a  resting  place  which  is  the  pleasurable  . 
tonus  level.  However,  in  the  event  that  there  is  too  much  stimuli  or  it  is  too 
complex,  unfamiliar,  or  the  degree  of  uncertainty  too  high,  the  individual  will  enter 
a  Zone  of  Anxiety  in  which  he/she  becomes  disinterested  and  inefficient. 
Performance,  it  would  follow,  is  negatively  affected.  Day  (1982)  writes  that 
"collative  variability  is  to  be  found  in  characteristics  of  stimuli  usually  labeled  with 
such  adjectives  as  complex,  novel,  incongruent,  difficult,  and  obscure"  (p.  21). 

Is  this  construct  separate  from  intelligence?  Day  (1968)  found  no 
significant  correlations  between  scores  on  his  Test  of  Specific  Curiosity  and  total 
intelligence  scores.  Langevin  (1971)  found  curiosity  measures  which  he  tested  to  be 
distinct  from  intelligence  on  the  whole.  The  arousal  model  put  forth  by  Day  and 
Berlyne  (1971)  suggests  that  intelligence  alone  is  inadequate  to  account  for 
classroom  achievement.  They  note  that  the  next  "most  potent  factor  is  probably 
motivation"  (p.  295).  Curiosity  is  motivation  which  is  intrinsic. 

Defining  curiosity  for  this  study.   Maw  and  Maw  (1966)  describe 
curiosity  as  an  arousal  state  in  which  the  individual  desires  to  know  more  about 
one's  self  or  environment.  Since  much  of  Maw  and  Maw's  work  built  upon  that  of 
Berlyne,  and  because  their  operationalized  definition  of  curiosity  involved  children, 
it  was  particularly  useful  to  the  present  study.  According  to  Maw  and  Maw  (1964), 
"...curiosity  is  demonstrated  by  an  elementary  school  child  when  he: 

1.  reacts  positively  to  new  strange,  incongruous,  or  mysterious 
elements  in  his  environment  by  moving  toward  them,  by  exploring, 
or  by  manipulating  them. 

2.  exhibits  a  need  or  a  desire  to  know  more  about  himself  and/or  his 
environment. 

3.  scans  his  surroundings  seeking  new  experiences. 

4.  persists  in  examining  and  exploring  stimuli  in  order  to  know  more 
about  them"  (p.  3D. 

Cecil,  Gray,  Thornburg,  and  Ispa  (1985)  extend  the  definition  of  Maw  and 
Maw  and  consider  curiosity  to  be  an  arousal  state  that  leads  to  and  is  a 
prerequisite  for  exploration,  play,  and  creativity.  Beswick  and  Tallmadge  (1971) 
describe  a  cognitive  process  theory*  °f  curiosity  in  which  curiosity  is  related  to 
conceptual  conflict.  In  this  theory,  "the  trait  of  curiosity  is  defined  as  an 
individual's  readiness  or  predisposition  to  seek,  maintain,  and  resolve  conceptual 
conflicts"  (p.  456).  In  proposing  a  model  of  teaching  and  learning,  another  author 
defines  curiosity  as  "the  individual's  desire  to  question  or  investigate"  (Parker  and 
Engel,  1984).  Penney  and  McCann  (1964)  make  a  distinction  between  curiosity  and 
reactive  curiosity  stating  that  a  child  may  be  curious  but  may  not  react  to  his  or 
her  curiosity.  The  definition  is  very  similar  to  that  of  Maw  and  Maw. 

Curiosity  as  trait  and  state.  The  literature  suggests  that  curiosity  is 
both  a  state  and  a  trait  variable  (e.g.,  Day,  1982).   As  a  state,  curiosity  is  a 
motivational  variable  which  varies  depending  on  the  situation  in  which  curiosity  is 
aroused  and  individual  differences.  As  a  trait,  curiosity  is  a  more  stable  variable 
indicating  individual  differences  in  one's  potential  to  experience  curiosity.  A  study 
by  Nay  lor  (1981),  gave  support  for  the  theoretical  distinction  between  state  and 
trait  curiosity.   According  to  Naylor,  people  who  are  higher  in  trait  curiosity 
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should  experience  curiosity  arousal  across  a  wider  range  of  situations  and  will  also 
have  the  capacity  to  experience  greater  intensities  of  state  curiosity.  Berlyne's 
theory  is  concerned  with  curiosity  as  a  state  of  arousal  brought  about  by  stimuli 
high  in  collative  variability  and  uncertainty.  The  theory  does  not,  however, 
overlook  the  importance  of  individual  differences  or  what  is  generally  termed  trait 
curiosity.  As  mentioned  earlier,  both  Day  and  Berlyne  (1971)  agree  that 
individuals  differ  in  tolerance  and  preference  for  arousal  potential:  "When 
confronted  by  a  specific  amount  of  arousal  potential,  some  people  will  react  with 
positive  affect,  interest,  and  exploration.  Others  may  become  overly  tense  and 
inefficient,  and  try  to  reject,  avoid,  or  withdraw  from  the  source  of  stimulation"  (p. 
304).  This  study  investigates  trait  curiosity.  The  relation  between  state  and  trait 
curiosity  should  therefore  be  important  to  educators  who  must  balance  the  amount 
of  stimulation  designed  to  arouse  curiosity  (the  motivational  state  of  curiosity)  with 
an  awareness  of  different  individuals1  tolerance  for  arousal  potential  (curiosity  as  a 
trait  variable). 

Specific  and  diversive  curiosity.  Curiosity  has  been  described  as  both  a 
state  and  a  trait,  arising  out  of  both  specific  and  diversive  exploration.  Berlyne's 
distinction  between  specific  and  diversive  exploration  was  built  upon  by  Day  who 
developed  tests  of  specific  and  diversive  curiosity  (Maw  &  Maw,  1977).  Generally, 
people  are  pre-disposed  to  traits  associated  with  being  specifically  or  diversively 
curious.  Referring  back  to  Maw  and  Maw's  (196^)  definition  of  curiosity  on  page 
10,  parts  1,  2,  and  4  of  their  definition  refer  to  specific  curiosity  while  part  3  refers 
to  diversive  curiosity. 

This  study  investigated  differences  among  children  in  trait  curiosity. 
However.,  in  as  much  as  the  two  learner  control  treatments  were  designed  with 
different  amounts  of  uncertainty  by  virtue  of  advisement  versus  no  advisement, 
these  conditions  can  be  thought  of  as  affecting  state  curiosity  which  should  affect 
performance  as  well.  Some  authors  (e.g.,  Necka,  1989)  discuss  strategies  designed 
to  nurture  trait  curiosity  through  instances  of  increased  state  curiosity.  Such 
strategies  included  brainstorming,  role  playing,  and  putting  forth  questions  of  the 
"I  wonder  if  really...?"  genre.  This  seems  to  indicate  that  perhaps  curiosity  as  a 
trait  (as  opposed  to  a  state)  can  be  enhanced  but  no  longitudinal  studies  of  this  sort 
could  be  found.  A  number  of  studies  cited  in  a  review  by  Maw  and  Maw  (1977) 
show  high  and  low  trait  curiosity  prevailing  under  certain  conditions  in  which  state 
curiosity  was  either  consistently  stimulated  or  modelled  or  it  was  not.  A  few  other 
studies  discussed  in  Maw  and  Maw's  review  included  strategies  for  raising  curiosity 
by  training  pupils  to  ask  questions. 

The  present  study  was  limited  to  investigating  the  effect  of  trait  curiosity 
on  learning  in  a  learner  control  environment. 

LEARNER  CONTROL 

The  Construct 

Defining  learner  control  for  this  study.  "Learner  control  '  as  a  general 
concept,  and  terms  which  are  related  to  learner  control,  such  as  "advisement," 
should  be  defined. 

This  stud  y  uses  Milheim  and  Azbeil's  (1988)  definition  of  learner  control  as! 

...the  degree  to  which  a  learner  can  direct  his  or  her  own  learning  process.  .[The  term)  most 
often  describes  the  instructional  choices  made  during  a  particular  le^on.  By  definition  these 
choices  can  be  made  either  by  the  instructional  program  <a*  ontfinalK  defined  hy  thr  HeMfmen 
or  hy  the  learner  during  the  presentation  of  materials  (p.  3) 
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A  lesson  which  is  highly  controlled  by  the  programmer  and  whose  path  and 
possible  choices  are  predetermined  is  generally  called  a  "designer  control"  or 
"program  control"  lesson.  A  lesson  which  allows  the  learner  to  control  options  such 
as  path  through  the  lesson,  choice  to  exit,  choice  to  be  remediated,  pacing  and  so  on 
is  called  a  "learner  control"  lesson.  One  which  affords  opportunities  for  exploration 
and  learner  control  but  with  advice  ranging  from  information  provided  about  the 
student's  progress  to  suggestions  for  sequencing  or  other  meaningful  learning 
advice  is  often  called  learner  control  with  advisement" 

Selected  Research  on  Learner  Control 

A  number  of  studies  show  that  positive  effects  have  been  achieved  with  the 
use  of  interactive  video  (e.g.,  Dalton,  1986;  Bosco  &  Wagner,  1988;  Hannafin  & 
Colamaio,  1988;  Abrams,  1986).  Of  particular  interest  are  the  studies  in  which 
learner  control  was  a  factor.  Some  studies  have  shown  positive  effects  related  to 
increased  learner  control  (e.g.,  Laurillard,  1984;  Merrill,  1975).  The  addition  of 
some  form  of  advisement  to  a  lesson  involving  learner  control  has  been  associated 
with  positive  results  in  increasing  achievement  (Tennyson,  1980;  Tennyson,  1984; 
Tennyson,  Christensen  &  Park,  1984).  Gay  (1986)  advises  that  the  appropriate 
level  of  learner  control  may  be  contingent  on  factors  such  as  individual  differences 
or  prior  conceptual  understanding  (Gay,  1986).  No  studies  on  learner  control 
looked  at  adults  or  children's  level  of  curiosity  as  a  possible  influencing  variable  on 
successful  learning  in  various  learner  control  situations.  Most  studies  have  used 
college  students  or  adults  as  subjects,  Hannafin  (1984)  notes  that  older  learners 
may  be  bettor  able  to  utilize  the  learner  control  options  because  of  their  refined 
cognitive  abilities.  Younger  subjects  (elementary  or  junior  high  school  age)  may  not 
yet  have  the  necessary  skills  to  make  the  most  of  such  a  lesson.  Arnone  and 
Grabowski  (1992)  addressed  ways  of  coping  with  this  developmental  challenge  by 
including  an  advisement  function  with  learner  control. 

The  Bridge  Study  Between  Learner  Control  and  Curiosity 

The  Arnone  and  Grabowski  study  (1992)  functioned  as  the  bridge  between 
some  of  the  earlier  research  on  learner  control  and  this  doctoral  dissertation.  For 
that  reason,  it  is  noted  as  the  "bridge  study"  for  the  present  work.  It  is  one  of  the 
few  studies  which  studied  young  subjects  (grades  1  and  2)  in  research  on  learner 
control.  The  study  found  a  significant  difference  in  achievement  between  subjects 
in  a  total  learner  control  lesion  and  subjects  in  a  learner  control  with  advisement 
lesson.  That  study  did  not,  however,  examine  trait  curiosity  and  its  potential 
influencing  effect  on  children's  learning  in  different  learner  control  situations.  Will 
a  more  curious  child  be  more  comfortable  about  exploring  in  a  learner  control 
environnie.it  than  the  less  curious  child?  Will  the  less  curious  child  show  higher 
performance  on  achievement  in  a  learner  control  environment  if  advisement  is 
provided?  Since  one  aspect  of  curiosity  is  willingness  to  explore  and  tolerance  for 
uncertainty  in  a  situation,  it  was  expected  that  there  would  be  overall  performance 
differences  between  high  and  low  curious  subjects  in  a  learner  control  lesson  and 
that  there  would  be  an  interaction  between  curiosity  level  and  the  learner  control 
treatment  (advisement  versus  no  advisement).  The  purpose  of  the  present  study 
was  to  extend  the  research  by  Arnone  and  Grabowski  and  to  explore  the  effect  of 
trait  curiosity  level  of  first  and  second  graders  on  achievement  in  a  learner  control 
lesson. 


8&  5, 


6 


PREDICTIONS 

The  research  predictions  must  be  prefaced  with  an  interpretation  of  how  the 
research  in  the  two  separate  areas  of  curiosity  and  learner  control  in  an  interactive 
learning  environment  can  be  merged  in  a  meaningful  way.  The  following 
discussion  proposes  one  way  of  thinking  about  this. 

The  Merger 

It  may  help  to  conceptualize  "learner  control"  as  a  continuum  with  no 
learner  control  on  the  extreme  left  and  a  high  amount  of  learner  control  on  the 
right.  No  learner  control  would  mean  no  opportunities  for  exploration  or  choices. 
Such  an  environment  would  essentially  shelter  the  learner  from  uncertainty  since 
the  programmer  has  basically  mapped  out  a  path  for  the  learner. 

On  the  opposite  end  of  the  continuum  is  a  high  degree  of  learner  control 
which  translates  into  many  opportunities  for  exploration  and  choice.  By  its  very 
nature,  high  learner  control  will  present  an  environment  which  is  less  familiar, 
contains  more  collative  variability  by  virtue  of  its  many  options,  and  has  a  much 
higher  degree  of  uncertainty.  The  learner  must  encounter  the  lesson  on  his  or  her 
own,  choose  to  go  down  a  certain  path  with  no  guide,  decide  whether  or  not  to  avail 
himself/herself  of  the  options  or  leave  them  be. 

The  mid-section  of  this  continuum  would  be  learner  control  with  advisement 
or  guidance.  In  this  case,  the  learner  would  experience  many  and  perhaps  all  the 
options  available  with  a  high  degree  of  learner  control  but  with  one  difference  -- 
advisement.  Exploration  would  take  the  form  of  more  guided  discovery.  In  essence, 
collative  .ariability  could  be  quite  high  but  the  uncertainty  associated  with  it  would 
be  reduced  through  the  guidance  which  helps  in  collation.  If,  as  Day  and  Berlyne 
(1971)  contend,  collative  variability  is  essential  to  arousing  curiosity,  but  that  too 
much  collative  variability  can  actually  have  the  effect  of  depressing  curiosity,  then 
the  midpoint  on  the  continuum  of  learner  control  should  be  favorable  towards 
increasing  performance  for  all  students,  and  particularly  favorable  for  low  curious 
students. 

In  the  curvilinear  relationship  between  arousal  and  efficiency  of 
performance  that  Day  (1982)  describes,  curiosity  is  a  dynamic  variable  which  can 
be  induced  or  manipulated,  while  this  study*  investigated  curiosity  as  a  stable  trait 
variable.  How  then  can  that  curve  be  interpreted  in  the  present  study?  Using 
prior  research  on  curiosity  as  support,  it  was  predicted  that  the  placement  of  an 
individual's  curve  on  the  arousal  axis  is  partially  dependent  on  how  curious  that 
individual  is  initially.  For  example,  one  would  expect  that,  in  the  presence  of 
stimuli  with  a  high  degree  of  uncertainty,  a  low  curious  student  may  enter  the  Zone 
of  Anxiety,  inefficiency,  or  disinterest  sooner  than  the  high  curious  child,  resulting 
in  a  negative  effect  on  one's  score  on  a  post  treatment  achievement  test.  The  low 
curious  child  in  any  environment  in  which  learner  control  was  a  factor  was 
predicted  to  score  lower  on  a  post  treatment  achievement  test  simply  because  there 
is  more  uncertainty  present  than  in  a  situation  in  which  there  was  no  learner 
control  (e.g.,  program  control).  However,  it  was  also  predicted  that  the  low  curious 
child  would  score  higher  in  a  learner  control  lesson  which  offers  advisement  than  in 
the  same  lesson  without  advisement. 

The  authors  argue  that  Learner  Control  with  Advisement  encourages 
movement  into  the  Zone  of  Curiosity  with  enough  collative  variability  to  be  exciting 
and  interesting  but  enough  advisement  strategies  to  reduce  the  potential  for 
anxiety  or  inefficiency  (in  terms  of  cognitive  processing)  associated  with  uncertainty 
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in  a  learner  control  environment.  Figure  1  illustrates  this  interpretation.  Higher 
achievement  test  scores  following  the  lesson,  therefore,  were  predicted.  This  effect 
was  expected  to  be  most  pronounced  with  low  curious  students  since  their  lower 
optimal  level  of  arousal  would  be  accommodated  by  the  guidance  afforded  in  the 
Learner  Control  with  Advisement  lesson,  in  essence,  keeping  them  in  the  Zone  of 
Curiosity  longer. 


ZONE  *1 


No  Learner  Control 

in.  ...  i  ii  m  .<- 


Learner  Control  with 
Advisement 


THE  LEARNER  CONTROL  CONTINUUM 


ZONE  *3 


High  Learner  Control 


u 


Low  Uncertainty 


Unmotivated 
Disinterested 


I      Moderate  Uncertainty 

|    Zone  of  Curiosity 

Approach 
I  Exploration 
|  Excitement 

Interest 


High  Uncertainty 


Avoidance 
Disinterested 
Inefficient 


Tonus  Level 


ACTIVATION 

(ARCLSAL) 


FIGURE  1  □  Learner  Control  and  Curiosity 
(Adapted  from  Day,  1 982) 


Predictions  for  performance  can  be  related  to  figure  1  by  noting  that  zone 
#1  was  marked  with  disinterest  and  lack  of  motivation,  zone  #2  with  approach  and 
interest,  and  zone  #3  with  avoidance  and  disinterest.  The  zones  reflect  a  curvilinear 
relationship  with  respect  to  performance.  Figure  2  shows  how  differences  in 
learner  control  which  range  from  a  low  to  high  degree  of  uncertainty  may  be 
experienced  differently  by  high  and  low  trait  curiosity  subjects.  The  numbers 
within  the  cells  refer  to  the  zones  described  above  and  in  Figure  1,  Arnone  and 
Grabowski  (1992)  found  significant  differences  between  learner  control  and  learner 
control  with  advisement  (high  and  moderate  uncertainty)  but  found  no  significant 
differences  between  program  control  (low  uncertainty)  and  the  other  treatments. 
Since  this  study  built  on  the  significant  differences  found  in  the  previous  study,  it 
examined  learner  control  without  advisement  and  learner  control  with  advisement 
(high  and  moderate  levels  of  uncertainty),  only. 
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TRAIT  CURIOSITY  LEVEL 
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Uncertainty 

Low 

High 

High 

3 

2  or  3 
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2  or  1 
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1 

FIGURE  2    □  Predictions  of  Performance  Based  on  interaction 
Between  Trait  Curiosity  and  Learner  Control 


Summary  of  Predictions 


1)  Learner  control  with  advisement  will  be  superior  to  learner  control 
without  advisement  in  promoting  achievement  in  young  learners. 

2)  High  curious  students  will  perform  better  than  low  curious  student.*  in  a 
learner  control  environment  regardless  of  treatment. 

3)  There  will  be  an  interaction  effect  between  curiosity  of  students  and 
lesson  treatment. 


METHOD 


This  study  built  on  the  previous  work  of  Arnone  and  Grabowski  (1992)  and 
utilized  the  same  lesson  materials  and  the  same  post  treatment  achievement 
instrument.  Same-age  subjects  within  the  same  school  district  assured  that  a  high 
degree  of  comparability  could  be  achieved  across  the  two  studies.    Since  the 
important  new  dimension  was  trait  curiosity,  it  was  important  to  hold  the  other 
factors  as  constant  as  possible. 

The  study  consisted  of  two  parts:  1)  a  series  of  three  pilot  studies  related  to 
the  testing  and  development  of  the  curiosity  instruments  and  2)  the  main  study 
involving  the  administration  of  the  curiosity  instrument,  the  intervention,  and  a 
post-treatment  achievement  test.  Additionally,  the  main  study  involved 
interviewing  a  random  sample  of  the  subjects  about  their  experience. 

Only  subjects  with  parental  approval  participated.  The  study  was  conducted 
in  a  school  setting  as  opposed  to  the  lab  which  afforded  some  veridicality  with  a 
real  world  learning  milieu. 

The  three  pilot  studies  were  conducted  in  the  months  before  the  conduct  of 
the  main  study  to  determine  the  appropriateness  of  four  curiosity  instruments  for 
the  sample.  The  first  two  pilot  studies  were  administered  to  18  children  each  in  a 
different  school  district  than  the  main  study.  The  third  pilot  study  took  place 
within  the  same  school  district  as  the  main  study  but  at  a  different  school  and 
involved  60  children.  In  addition  to  providing  the  basis  for  selecting  the  curiosity 
instruments  to  be  used  in  the  main  study,  these  pilots  also  contributed  to  fine* 
tuning  administration  procedures  prior  to  the  main  study. 
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Subjects 


A  total  of  84  first  and  second  grade  students  from  a  suburban  school  district 
in  Central  New  York  participated  in  the  main  study.    All  84  subjects  were 
administered  the  preliminary  curiosity  measure,  received  the  intervention,  and  took 
the  achievement  test.  However,  two  subjects  of  the  84  were  dropped,  one  because  of 
computer  malfunction  (audio  drop-out  which  lasted  several  minutes  during  the 
lesson  treatment)  and  one  because  of  administrator  error  (administrator  did  not  set 
up  computer  for  the  graphic  overlays).  A  total  of  82  subjects  remained. 

The  scores  of  the  82  subjects  on  the  curiosity  measure  were  collapsed  into  a 
categorical  variable  with  two  levels  of  curiosity.  On  a  20  item  test,  36  subjects 
scored  high  (M  =  7.19),  11  subjects  scored  in  the  middle  (M  =  12,81)  and  35  subjects 
scored  low  {M  =  6.26).  As  planned,  the  middle  scorers  were  dropped  from  the 
analysis  leaving  71  subjects.  The  difference  between  the  extreme  scorers  was 
significant  at  £  <  .0001, 

There  were  36  high  curious  subjects  and  35  low  curious  subjects.  There 
were  31  males  and  40  femalos.  Thirty-nine  subjects  were  first  graders  while  32 
subjects  were  second  gTaders.  Sixteen  first  grade  subjects  were  high  curious  while 
23  first  grade  subjects  were  low  curious.  Twenty  second  grade  subjects  were  high 
curious  while  12  second  grade  subjects  were  low  curious.  Among  first  grade 
subjects,  25  were  female  and  14  were  male;  among  second  grade  subjects,  15  were 
female  and  17  were  male.  First  graders  ranged  in  age  from  6  years  1  month  to  7 
years  3  months.  Second  graders  ranged  in  age  from  7  years  3  months  to  8  years  6 
months. 


Materials  as  input  to  the  study  consisted  of  the  curiosity  assessment 
instruments,  the  lesson,  and  the  two  learner  control  treatments.  Materials  as 
output  consisted  of  the  post  treatment  achievement  test,  lesson  log,  and  interview 
protocol.  In  this  section,  the  lesson  and  the  two  learner  control  treatments  are 
discussed.  The  other  materials  are  discussed  under  'Instrumentation." 

Lesson  Description 

The  lesson  content  was  the  same  as  that  used  in  the  bridge  study,  A 
subject  matter  expert  in  art  education  provided  the  content  for  the  treatments.  The 
lesson  was  designed  as  a  visit  to  an  art  museum.  The  subject  matter  expert  had 
previously  served  as  Curator  of  Education  for  the  museum  and  it  was  possible  to 
acquire  many  visuals  for  the  treatments. 

The  lesson  itself  included  a  general  introduction  and  three  segments  on 
ceramics,  sculptures,  and  paintings.  The  content  involved  both  facts  and  concepts 
and  the  lesson  provided  opportunities  for  practice,  feedback,  and  remediation. 

The  lessons  were  designed  for  subjects  with  limited  reading  experience  and 
contained  no  text.  A  narrator  was  used  where  text  would  have  been  necessary. 
The  use  of  a  touch  screen  in  place  of  a  keyboard  further  simplified  the  young 
learners'  task  of  responding. 

An  interactive  video  lesson  was  developed  to  be  delivered  on  a  Sony 
Laservision  videodisc  player  interfaced  with  a  MS-DOS  compatible  computer,  a 
touch-sensitive  screen,  and  headsets.  The  lesson  was  programmed  using  ICON 
Author.  A  combination  of  motion  video,  slides,  and  computer  graphics  was  used  in 
the  presentation. 
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CBIV  Instructional  Treatments 

The  treatments  were  labelled  1)  Learner  control  without  advisement  (No 
Advisement)  and  2)  Learner  control  with  advisement  < Advisement).  Both 
treatments  contained  the  same  essential  content  and  both  provided  opportunities 
for  practice,  remediation,  and  feedback   Two  instructional  designers  provided 
judgments  on  the  appropriateness  of  the  treatments  in  reflecting  the  constructs  of 
learner  control  with  advisement  and  learner  control  without  advisement.  The 
dissertation  study  did  involve  one  change  in  the  treatments.  Rather  than  receiving 
two  opportunities  to  respond  to  practice  items,  the  lessons  were  re-programmed  so 
that  subjects  were  provided  the  opportunity  for  remediation  or  advisement  after  one 
incorrect  response.  The  change  was  made  in  an  attempt  to  strengthen  the 
advisement  treatment. 

The  average  time  spent  in  the  learner  control  with  advisement  lesson  was 
13.2  minutes  while  the  average  time  spent  in  the  learner  control  without 
advisement  lesson  was  15.4  minutes.  This  did  not  include  the  time  subjects  spent 
in  a  pre-lesson  warm-up  which  averaged  3.26  minutes  or  time  spent  in  taking  the 
post-treatment  achievement  test. 

Learner  control  without  advisement.  In  the  learner  control  without 
advisement  group,  subjects  were  given  the  opportunity  to  sequence  the  material  in 
any  way  they  chose.  Whether  to  review  segments  where  practice  items  had  been 
missed  was  also  a  decision  left  to  these  subjects.  Students  had  the  opportunity  to 
omit  entire  sections  or  sub-sections,  or  opt  out  of  the  lesson  altogether,  if  they  so 
desired.  Students  controlled  their  own  pace  through  the  lesson.  Additionally, 
subjects  could  freeze  images  on  the  screen.  This  was  called  a  "Stop  and  Look" 
routine  since  whenever  a  particular  icon  was  present,  the  subject  could  freeze  the 
image  on  the  screen  to  explore  it  more  closely.  Subjects  were  familiar  with  the  icon 
since  they  were  exposed  to  the  icon  and  its  meaning  in  the  pre-lesson  practice 
session. 

Learner  control  with  advisement.  Subjects  in  the  learner  control  with 
advisement  group  received  the  same  opportunities  to  explore  the  lesson  as  the 
Learner  Control  Without  Advisement  group.  However,  certain  "advisement" 
strategies  were  also  employed  in  this  group  which  provided  some  guidance  when 
poor  decisions  were  made  and  also  encouraged  guided  discovery,  For  example,  if  a 
student  decided  to  skip  out  of  a  section,  he/she  received  this  ndvisement:  "Are  you 
sure  you  want  to  end  the  lesson?  This  next  section  is  very  interesting.  You  might 
really  enjoy  it"  or  "Aren't  you  going  to  wonder  about  what  you'll  be  missing"  (Inn1 
was  taken  not  to  instill  fear  (related  to  the  expectation  of  being  tested)  uk  a 
motivation  to  continue  the  lesson  since  such  motivation  is  considered  extrinsic  and 
not  compatible  with  the  construct  of  curiosity  under  study. 

While  subjects  in  the  learner  control  without  advisement  group  could  take 
advantage  of  the  "Stop  and  Look"  routine  whenever  they  saw  the  associated  icon 
appear  on  the  screen,  subjects  in  the  learner  control  with  advisement  group  were 
advised  by  the  narrator  to  take  advantage  of  the  "Stop  and  Look"  routine  to  explore 
the  images  more  closely.  The  final  type  of  advisement  was  in  the  form  of  a  "Stop 
and  Think"  routine  which  generally  was  preceded  by  a  question  to  arouse  curiosity, 
For  example,  after  presenting  some  interesting  information  about  a  painting  that 
generally  intrigued  young  children,  this  question  was  posed:  "Do  you  wonder  how 
you  can  tell  this  from  looking  at  the  painting?  'Stop  and  Think1  about  it!  Then, 
touch  the  screen  when  you  are  ready  to  find  out."  All  audio  and  visuals  froze  at 
that  point  and  resumed  when  the  subject  touched  the  screen  to  proceed.  The 
subject  was  in  control  of  how  long  to  "Stop  and  Think;"  if  the  subject  wanted  to 
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spend  no  time  in  this  mode,  all  he/she  had  to  do  was  to  immediately  touch  the 
screen  as  soon  as  the  narrator  finished  speaking.  Other  than  the  advisement,  the 
instructional  content  remained  the  same  as  the  other  treatment. 

Instrumentation 

Curiosity  Instruments 

The  two  measures  which  were  selected  for  use  in  the  main  study  were: 

--Maw  and  Maw's  Which  To  Discuss  Test  (Maw  &  Maw,  1964)  with  a 
practice  session  and  administration  directions  tailored  to  primary  grades 

--Arnone  and  Sciretta's  Specific  Curiosity  Scale  for  Primary  Grades 
(SCSPG)  (Arnone  &  Sciretta,  1991) 

The  Which  To  Discuss  Test  does  not  require  reading  ability  and  is  a  test  of 
specific  curiosity  which  was  appropriate  for  the  study.  It  is  comprised  of  20  pairs  of 
geometric  symbols  and  figures.  One  symbol  is  more  familiar  and  /or  balanced  than 
the  other  in  each  of  the  pairs.  Preference  for  the  unbalanced  or  the  unfamiliar, 
according  to  Maw  and  Maw  (1964)  taps  into  the  part  of  the  definition  of  curiosity  in 
which  "a  child  manifests  curiosity  to  the  extent  that  he  'reacts  positively  to  new, 
strange,  incongruous,  or  mysterious  elements  in  his  environment  by  moving 
towards  them,  by  exploring  them,  or  by  manipulating  them/"  (p,  112).  Using  the 
M  ann-Whitney  U  test,  this  measure  showed  significant  differences  between  the 
means  of  the  high  and  low  curious  fifth  grade  children  supporting  the  hypothesis 
that  high  curious  children  select  the  unbalanced/unfamiliar  more  often  than  low 
curious  children.  Its  reliability  is  high  at  .91  computed  by  the  split-half  method; 
Silverstein  et  al  (1981)  used  Cronbach's  alpha  to  measure  the  test's  internal 
consistency  and  calculated  it  to  be  .90. 

The  Which  to  Discuss  Test  provided  the  basis  of  assignment  into  curiosity 
groups  as  it  correlated  positively  with  teacher  judgment  of  curiosity  in  the  pilot 
study,  and  because  Maw  and  Maw  had  previously  validated  the  instrument  on 
elementary  school  children.  It  also  had  higher  reliability  than  the  newly  developed 
SCSPG.  In  the  main  study,  the  Which  To  Discuss  Test  received  the  same  reliability 
coefficient  using  Cronbach's  alpha  as  it  did  in  the  pilot  studies  {n  =  84,  r  =  .90). 
The  reliability  coefficient  of  the  SCSPG  using  Cronbach's  alpha  was  r  =".58  in  the 
main  study.  The  Which  To  Discuss  Test  showed  a  significant  positive  correlation 
with  the  SCSPG  (r  =  .29,  £  =  .008).  Since  the  SCSPG  contained  items  which 
encompassed  all  the  aspects  of  specific  curiosity  and  the  Which  To  Discuss  Test 
focused  on  only  one  aspect  of  specific  curiosity,  another  correlation  was  computed. 
This  analysis  involved  correlating  only  those  items  of  the  SCSPG  which 
represented  the  aspect  of  curiosity  measured  in  the  Which  To  Discuss  Test  (i.e., 
preference  for  the  unfamiliar)  with  the  total  score  of  the  Which  To  Discuss  Test. 
This  time  the  correlation  was  much  higher  (r  =  .48,  £  =  .0001). 

Another  reason  for  choosing  Maw  and  Maw's  test  for  the  purposes  of 
grouping  is  that  it  focused  on  the  aspect  of  curiosity  (preference  for  the 
unfamiliar/willingness  to  explore)  most  related  to  the  experimental  treatments. 
The  primary  purpose  of  administering  the  SCSPG  in  the  main  study  was  to  add 
further  construct  validity  to  Maw  and  Maw's  scale  and  to  provide  one  more 
opportunity  for  refinement  and  analysis  of  the  SCSPG. 
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Scoring  for  curiosity  instruments.  In  the  Which  Po  Discuss  Test,  the 
child  received  1  point  for  each  item  which  showed  a  preference  for  the  unfamiliar 
and  0  points  for  items  which  did  not  indicate  this.  The  total  score  was  the  sum  of 
points  received.  The  Specific  Curiosity  Scale  for  Primary  Grades  was  scored  in  the 
same  way,  that  is,  subjects  received  a  1  for  responses  which  indicated  specific  trait 
curiosity  and  a  0  for  responses  which  did  not.  The  minimum  total  score  a  subject 
could  receive  was  0  and  the  maximum  total  score  possible  was  20  for  each  of  the 
two  measures.  The  actual  scores  ranged  from  a  low  of  0  to  a  high  of  20  for  the 
Which  To  Discuss  Test  and  a  low  of  6  and  a  high  of  19  on  the  SCSPG. 

Achievement  Instrument 

The  present  study  used  the  same  achievement  test  which  was  used  in  the 
bridge  study  although  it  was  revised  for  ease  of  administration.  The  subject  matter 
expert  reviewed  the  original  achievement  test  for  content  validity  and  determined 
that  it  adequately  represented  the  content  presented  in  the  lesson.  A  content  table 
of  specifications  was  used  to  derive  the  items  for  the  achievement  instrument  and 
to  insure  that  the  items  were  weighted  in  accordance  with  their  importance  in  the 
lesson  itself. 

Part  of  the  challenge  of  designing  materials  to  measure  the  effectiveness  of 
the  treatments  had  to  do  with  the  type  of  content  to  be  presented.  "Pat"  answers 
are  inappropriate  in  an  area  such  as  art  education  and  museums.  In  measuring 
achievement,  it  was  necessary  to  devise  an  instrument  which  gave  the  child  the 
opportunity  to  be  more  expressive  while  demonstrating  that  he/she  had  indeed 
acquired  the  new  information.  Although  the  bridge  study  also  included  an 
instrument  to  measure  whether  curiosity  had  been  aroused  more  by  one  treatment 
than  another,  the  present  study  looked  only  at  achievement. 

The  achievement  test,  then,  consisted  of  8  items  which  measured  whether 
learning  of  the  content  had  occurred.  A  number  of  points  could  be  accrued  for  each 
item.  To  control  for  order  effect  and  fatigue,  the  computer  was  programmed  to 
randomly  generate  the  order  of  test  items  for  each  subject.  Because  of  the  open- 
ended  nature  of  the  questions,  it  was  found  in  the  bridge  study  to  be  necessary  to 
distinguish  between  responses  which  were  specific  to  the  lesson  (lesson-related)  and 
those  which  were  not  (lesson-unrelated).  This  study  also  used  the  same  method  in 
differentiating  responses. 

Scoring  for  achievement  instrument.  In  the  achievement  measure,  the 
subject  was  presented  video  still  images  of  aspects  of  the  museum  encountered  in 
the  lesson  such  as  paintings,  sculptures,  and  ceramics.  For  each  item,  the 
administrator  requested  the  subject  to  tell  her  everything  the  subject  knew  about 
the  picture.  Responses  were  recorded  on  a  paper  and  pencil  instrument  and  scored 
later.  After  responding,  the  subject  touched  the  screen  to  proceed  to  the  next 
image.  The  instrument  was  devised  to  take  somewhat  qualitative  data  and  give  it 
points  based  on  1)  recall  of  facts  and  concepts  introduced  in  the  treatment  and  2) 
number  of  appropriate  responses.  The  minimum  possible  score  was  0  if  the  child 
gave  no  appropriate  responses.  The  maximum  possible  score  was  derived  from  a 
content  analysis  of  the  lesson.  There  were  102  possible  acceptable  responses  which 
were  considered  lesson-related.  The  reason  for  the  high  number  of  potential 
responses  was  due  to  the  open-ended  nature  of  the  instrument.  The  actual  scores 
ranged  from  a  low  of  3  to  a  high  of  40. 
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Interview  Protocol 

Twenty-five  subjects  were  randomly  selected  for  brief  interviews  following 
the  achievement  test  to  garner  qualitative  information  about  the  subjects1 
experience  in  a  learner  control  environment  and  about  decisions  they  made  such  as 
whether  or  not  to  exit.  Twenty  interviews  were  subsequently  transcribed  for 
analysis.  Three  of  the  original  25  interviews  were  inaudible  and  two  subjects  who 
had  been  interviewed  scored  in  the  middle  curiosity  group  and  thus  were  not  in  the 
sample  for  analysis  purposes.  Fourteen  interviews  were  conducted  by  one  of  the 
authors  and  the  remaining  interviews  were  split  up  between  the  administrators. 

Through  random  chance,  1 1  subjects  interviewed  were  male  and  9  were 
female,  12  subjects  were  high  curious  and  8  were  low  curious,  10  subjects  were  first 
graders  and  10  subjects  were  second  graders.  Nineteen  subjects  had  received  the 
Advisement  lesson  while  1  subject  had  received  the  No  Advisement  lesson.  The 
subject  who  received  the  No  Advisement  lesson  was  a  first  grade  high  curious 
female. 

Lesson  Log 

The  lesson  log  was  completed  by  the  administrator  while  the  subject  was 
engaged  in  the  lesson.  The  administrator  observed  certain  decisions  made  by  the 
subject  and  recorded  the  data  on  the  lesson  log.  The  information  included  data  on 
sequencing  decisions,  exiting  decisions,  remediation  decisions,  reaction  to 
advisement,  and  how  many  times  each  subject  used  the  "Stop  and  Look"  learner 
control  option. 

Scoring  for  lesson  log.  The  data  for  the  lesson  logs  were  recorded  by  the 
administrators  on  the  logs.  The  number  of  times  each  subject  made  the  various 
decisions  were  later  tallied. 

Procedures 

A  number  of  procedures  were  related  to  the  conduct,  setting,  and  situational 
organization  of  the  study.  Additionally,  certain  procedures  and  precautions  were 
undertaken  to  protect,  the  study's  internal  validity. 

Administration 

Curiosity  measures.  One  month  before  the  subjects  in  the  main  study 
participated  in  the  CBIV  lesson,  they  were  administered  curiosity  measures  to 
determine  their  trait  curiosity  level  (specific).  These  measures  were  administered  by 
one  of  the  authors  in  a  group  setting  in  the  actual  classrooms  of  participating 
subjects.  The  data  for  the  two  measures  were  collected  over  four  days  within  a  one- 
week  time  period.  Each  instrument  was  administered  to  one  classroom  at  a  time. 
The  six  teachers  participating  in  the  study  determined  the  schedule  for 
administration  in  their  respective  classrooms.  The  Which  To  Discuss  Test  was  the 
first  instrument  administered  and  the  SCSPG  was  the  second.  The  tests  were 
administered  on  different  days  to  control  for  fatigue  as  a  confounding  variable.  The 
reason  for  the  order  of  administration  of  the  two  measures  was  that  the  SCSPG 
required  more  time  to  administer  and  raoie  listening  skills  on  the  part  of  the 
students.  Since  these  subjects'  test  taking  experience  was  limited  (especially  the 
first  graders)  the  decision  was  made  to  give  them  the  less  demanding  test  first. 

Subjects'  curiosity  scores  were  collapsed  into  the  categorical  variable  of  high 
or  low  curiosity  and  were  assigned  to  either  the  learner  control  without  advisement 
treatment  or  the  learner  control  with  advisement  treatment.  Block  randomization 
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on  the  curiosity  variable  was  used  to  accomplish  this.  As  mentioned  earlier,  the 
middle  scorers  (n  =  11)  were  dropped  in  deriving  the  high  and  low  groups  and  an 
independent  means  t-test  showed  the  difference  between  the  high  and  low  groups  to 
be  significant  at  the  p  =.0001  level.  To  test  that  there  were  no  significant 
differences  between  the  highs  in  their  assignment  to  the  two  treatment  conditions, 
and  likewise  between  the  lows,  ANOVA  was  used.  Results  of  this  analysis 
indicated  that  there  were  no  significant  differences  between  the  two  treatment 
groups  after  being  assigned  the  high  curious  subjects,  £  =  .77.  There  were  also  no 
significant  differences  between  the  two  treatment  groups  after  being  assigned  the 
low  curious  subjects,  £=  .85. 

Treatments.  Subjects  were  administered  the  treatments  individually  and 
received  a  pre-lesson  warm-up  to  familiarize  them  with  how  to  use  the  touch  screen 
and  to  introduce  several  icons  which  the  subject  would  encounter  during  the 
treatment.  The  achievement  test  was  administered  immediately  following  the 
treatment.  Audiotaped  interviews  of  randomly  selected  subjects  were  conducted 
following  the  achievement  test.  As  the  concluding  activity  before  being  returned  to 
class,  each  subject  viewed  the  same  brief  entertaining  video  at  a  separate  laserdisc 
station.  It  lasted  approximately  three  minutes.  This  activity  was  meant  to  serve 
as  a  distractor  from  the  main  content  (and  learner  control  condition)  of  the  lessons 
to  help  protect  the  study  from  the  potential  for  contamination.  Since  the  video  was 
the  last  thing  each  saw,  it  was  likely  to  be  the  first  and  most  discussed  aspect  of 
their  experience. 

Three  treatment  administrators  were  trained  during  a  half-day  session  two 
weeks  prior  to  the  study.  By  employing  three  administrators  to  administer 
treatments  it  was  possible  for  three  subjects  to  participate  simultaneously  at 
separate  interactive  video  stations.  The  interactive  video  stations  were  located  in  a 
large  room  within  the  school  the  subjects  attended.  Large  dividers  separated  each 
station  insuring  each  subject's  individual  effort  and  decreasing  distractive  elements. 

All  administrators  were  paid  a  daily  fee  for  both  the  training  day  and  the 
administration  days.  A  total  of  six  administration  days  over  a  two-week  period  was 
necessary  to  administer  the  treatments,  achievement  tests,  and  to  conduct 
interviews. 

Procedures:  Protection  of  Internal  Validity 

A  number  of  precautionary  administration  procedures  were  undertaken  in 
the  bridge  study,  to  protect  the  study's  internal  validity.  These  provisions  were 
replicated  in  the  present  study.  Subjects  were  blind  to  which  treatment  they  were 
receiving.  Both  treatments  were  interactive.  Administrators  were  hired  for  the 
treatments  and  post  test  to  control  for  experimenter  bias.  Measures  were  taken  to 
control  for  distractions  during  treatments.  Administrators  participated  in  a  half- 
day  training  session  in  which  they  eventually  reached  1007r  inter-rater  agreement 
on  reporting  the  data  during  a  beta  test.  All  treatment  administrators  were  blind 
to  the  hypotheses  of  the  study.  No  one  had  any  prior  knowledge  of  the  curiosity 
ratings  of  any  child  before  administering  the  treatment  and  had  no  idea  about  any 
of  the  expected  interactions.  Because  administrators  had  to  load  the  designated 
lesson  on  the  computer  for  each  subject,  they  necessarily  had  knowledge  of  which 
lesson  they  were  administering,  however. 
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Research  Design 

To  test  the  predictions  that  a)  subjects  in  the  Learner  Control  With 
Advisement  treatment  (Advisement)  will  score  significantly  higher  on  a  posttest  of 
achievement  tTian  subjects  in  the  Learner  Control  Without  Advisement  treatment 
(No  Advisement),  and  b)  high  curiosity  subjects  will  score  significantly  higher  on  a 
posttest  of  achievement  than  low  curious  subjects  regardless  of  treatment,  a  two- 
way  analysis  of  variance  was  conducted.  An  alpha  level  of  £  <  .01  was  selected. 

To  test  the  prediction  that  there  will  be  an  interaction  between  curiosity  of 
students  and  lesson  treatment,  pairwise  multiple  comparisons  were  made  choosing 
a  .05  level  of  significance.  These  follow-up  analyses  utilized  the  Newman-Keuls 
procedure. 

This  analysis  entailed  partitioning  the  continuous  variable  of  curiosity  into 
the  arbitrary  categories  of  high  and  low  curiosity.  According  to  Pedhazur  (1973), 
there  is  danger  in  categorization  since  it  can  lead  to  a  loss  of  information  resulting 
in  a  less  sensitive  analysis.  In  essence,  all  students  who  were  placed  in  one  group 
or  the  other  were  treated  as  if  they  had  the  same  score.  So  that  the  difference  of 
just  one  point  would  not  be  the  deciding  factor  in  group  assignment,  the  three 
middle  scores  were  eliminated.  As  reported  earlier,  this  resulted  in  a  loss  of  11 
subjects  but  a  highly  significant  p  value  in  differentiating  the  two  groups  (p  = 
.0001).  All  84  subjects  were  administered  the  treatments,  however,  as  not  doing  so 
would  have  raised  suspicions  (and  anger)  among  the  subjects  left  out  of  this  phase 
of  the  study. 

Block  randomization  of  subjects  to  the  treatment  groups  was  used  to 
control  for  differences  in  prior  knowledge  of  museums,  ability  level,  and  other 
factors. 

RESULTS 

Analysis 

A  2  X  2  analysis-of-variance  factorial  design  was  computed  to  test  whether 
performance  on  the  achievement  test  was  related  to  the  treatment,  curiosity,  or  the 
interaction  of  treatment  and  curiosity.  Since  there  were  uneaual  numbers  in  the 
cells,  the  general  linear  models  procedure  was  used.  The  results  indicated  that 
there  was  overall  significance  in  the  model,  £(3,67)  =  3.54,  £  =  .0191.  There  was  a 
significant  main  effect  for  curiosity  (j)  <  .01).  There  was  no  main  effect  for 
treatment.  No  significant  interaction  was  found.  Results  of  this  analysis  are 
reported  in  Table  1.  Type  III  Sums  of  Squares  are  reported  in  the  table  since 
unequal  sample  sizes  were  used. 

Subjects  in  the  Advisement  treatment  scored  only  marginally  and  non- 
significantly  higher  (M  =  15.62)  than  subjects  in  the  No  Advisement  treatment  (M  = 
15.22).  The  hypothesis  that  subjects  in  the  Learner  Control  with  Advisement 
treatment  would  score  significantly  higher  overall  than  subjects  in  the  Learner 
Control  Without  Advisement  treatment  was  not  supported.  Table  2  presents  the 
means,  standard  deviations,  and  range  of  scores  on  the  achievement  test  for  the  two 
treatments.  The  standard  deviations  were  virtually  the  same  for  each  treatment. 
The  range  of  scores  was  quite  similar  in  both  treatments  as  well.  While  there  was 
no  overall  significant  difference  between  treatments,  there  was,  however,  a 
significant  interaction  between  grade  level  and  treatment,  F  (1,63)  =  4.43,  ]>  <  .05. 
Post  hoc  comparisons  using  the  Newman-Keuls  Test  showed  that  first  graders  in 
the  Advisement  treatment  (n  =  18,  M  =  16.39)  scored  significantly  higher  than  first 
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grade  students  in  the  No  Advisement  treatment  (n  =  21,  JM  =  12.33).  The  opposite 
was  true  for  the  2nd  grade  subjects  with  those  receiving  No  Advisement  (n  =16t  M 
=  19.00)  performing  significantly  better  than  those  receiving  Advisement  (n  =  16,  M 
=  14.75).  Figure  3  illustrates  this  interaction. 

Table  3  presents  the  descriptive  statistics  for  the  achievement  test  smre 
based  on  curiosity.  The  difference  in  mean  scores  on  achievement  between  tne  two 
groups  is  quite  large  (difference  =  5.03  points)  in  favor  of  the  high  curiosity  group. 
Therefore,  the  hypothesis  that  high  curious  subjects  would  perform  better  in  an 
interactive  learner  control  lesson  than  low  curious  subjects  regardless  of  treatment 
was  supported.  Overall,  high  curious  subjects  received  significantly  higher  scores 
on  the  achievement  test  (M  =  17.89)  than  low  curious  subjects  (M  =  12.86).  The 
range  of  scores  also  reflects  a  dramatic  difference  with  the  highest  score  in  the  high 
curiosity  group  and  the  lowest  score  in  the  low  curiosity  group. 

Table  4  presents  the  descriptive  statistics  for  the  achievement  test  scores  by 
treatment  and  curiosity  groups.  High  curious  subjects  performed  slightly  better, 
although  not  significantly,  in  the  No  Advisement  treatment  (M  =  18.39)  than  they 
did  in  the  Advisement  treatment  (M  =  17.39).  Also,  high  curious  subjects  in  the  No 
Advisement  treatment  scored  significantly  higher  (M  =  18.39)  than  low  curious 
subjects  in  the  Advisement  treatment  (M  =  13.63)  at  the  jd  <  .05  level  with  the 
Newman-Keuls  test.   Low  curious  subjects  did  score  higher  in  the  Advisement 
treatment  (M  =  13.63)  than  low  curious  subjects  in  the  No  Advisement  treatment 
(M  =  12.21)  but  the  difference  was  not  significant  at  the  £  <  .05  level  (see  Figure  4). 

There  were  no  main  effects  for  grade  level  although  second  graders  achieved 
higher  scores  (n  =  32,  M  =  16.68)  *han  first  graders  (n  =  39,  M  =  14.20)  on  the 
achievement  test.  There  was  also  no  significant  interaction  between  grade  level 
and  curiosity  on  performance  on  tl.<s  achievement  test  suggesting  that  curiosity  was 
not  affected  by  the  higher  level  of  verbal  and  communication  skills  of  the  second 
graders. 

There  was  no  significant  main  effect  for  gender.  Males  and  females 
performed  equally  regardless  of  which  lesson  treatment  they  received.  Most 
importantly,  however,  no  interaction  between  gender  and  curiosity  suggests  that 
curiosity  is  not  affected  by  gender  differences.  High  curious  females  (n  =  18)  scored 
only  slightly  higher  (M,  =  18.17)  than  high  curious  males  (n  =  18,  M  =  17.61)  while 
low  curious  males  (n  =  13)  scored  slightly  higher  (M  =  13.15)  than  low  curious 
females  (n  =  22,  M  =  12.68). 


Table  1 


Analysis  of  Variance  Tests  of  Hypotheses 


Source 

df 

SS 

MS 

F 

Curiosity 

1 

436.91 

436.91 

9.77* 

Treatment 

1 

0.76 

0.76 

0.02 

Curiosity'Treatment 

1 

25.77 

25.77 

0.58 

Error 

67 

2997.16 

44.74 

C<  .01 
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Table  2 

Achievement  Test  Scores:  Descriptive  Statistics  by  Treatment  Group 


Variable 

N 

Mean 

Std.  Dev. 

Minimum 

Maximum 

(treatment) 

Score 

Score 

No  Advisement 

37 

15.22 

7.17 

4.00 

40.00 

Advisement 

34 

15.62 

7.00 

3.00 

38.00 

20  


I0  "  

NO 

ADVISEMENT  ADVISEMENT 


FIGURE  3  □   Mean  Achievement  Scores  by 
Treatment  and  Grade  Level 

Table  3 

Achievement  Test  Scores:  Descriptive  Statistics  bv  Curiosity  Groups 


Variable 

N 

Mean 

Std.  Dev. 

Minimum 

Maximum 

(Curiosity) 

Score 

Score 

Low 

35 

12.86 

5.98 

3.00 

29.00 

High 

36 

17.89 

7.19 

8.00 

40.00 

97 
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Table  4 

Achievement  Test  Scores:  Descriptive  Statistics  by  Treatment  and  Curiosity  firnups 


Variable 

N 

Mean 

Std.  Dev. 

Minimum 

Maximum 

Treatment  Curiosity 

Score 

.  Score 

No  Advisement  *-ow 

19 

12.21 

5.58 

4.00 

23.00 

High 

18 

18.39 

7.43 

8.00 

40.00 

Advisement  *-ow 

16 

13.63 

•  6.52 

3.00 

29.00 

High 

18 

17.39 

7.11 

8.00 

38.00 
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FIGURE  *  □   Mean  Achievement  Scores  by 
Treatment  and  Curiosity 
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In-Treatment  Data;  Time  on  Task,  Curiosity,  and  Treatment 

Time  on  task  is  the  actual  length  of  time  the  subject  spent  in  the  lesson; 
therefore,  the  time  spent  in  the  pre-lesson  warm-up  or  the  time  spent  in 
administration  of  the  achievement  test  was  not  included.  After  the  subject  received 
a  pre-lesson  warm-up  which  averaged  3.26  minutes,  the  narrator  instructed  the 
subject  to  "touch  the  screen  when  you  are  ready  to  begin  your  visit."  The  computer 
tallied  time  in  treatment  from  the  moment  the  subject  touched  the  screen  to  the 
conclusion  of  the  treatment.  Due  to  computer  error,  9  subjects'  time  data  were  lost. 
To  determine  if  there  was  a  relationship  between  time  on  task  and  treatment, 
curiosity,  and  grade  level,  an  analysis  of  variance  was  conducted.  The  analysis 
indicated  that  there  was  overall  significance  in  the  model,  £(7,54)  =  3.94,  £  <  .01. 
Significant  main  effects  existed  for  treatment  (j>  <  .05)  but  not  for  curiosity  or  grade 
level  at  the  j>  <  .05  level.  Subjects  in  the  Advisement  treatment  (n  =  34,  M  =  15.4) 
spent  approximately  two  minutes  longer  in  the  lesson  than  subjects  in  the  No 
Advisement  treatment  (n  =  28,  M  =  13.2).  No  significant  interactions  existed  at  the 
£  <  .05  level.  Figure  5  shows  the  interaction  of  curiosity  and  treatment  on  time  on 
task. 


FIGURE  5  □    Mean  Time  on  Task:  Interaction  of 
Curiosity  and  Treatment 

In -Treatment  Data:  Learner  Control  Decisions 

In  addition  to  time  on  task,  in-treatment  data  collected  included  decisions 
made  by  the  subject  related  to  other  learner  control  options  such  as  sequencing, 
remediation,  "Stop  and  Look,"  and  exiting,  as  well  as  subjects'  response  to 

advisement. 

Sequencing,  Most  subjects  availed  themselves  of  the  opportunity  to 
sequence  their  material.  Eighty-nine  percent  (89%,  n  =  63)  of  the  subjects  chose  a 
sequence  of  content  presentation  different  from  the  sequential  order  of  the  menu 
options.  Only  11%  (n  s  8)  of  the  subjects  chose  the  sequential  order  of  menu 
options.  Thirty-seven  and  one  half  percent  (37.5%,  n  =  3)  of  these  children  were  in 
the  high  curious  group  while  62.5%  (n  =  5)  were  in  the  low  curious  group. 

Exiting.  Almost  all  of  the  subjects  (87%,  n  =  62)  stayed  in  the  lesson.  Of 
the  13%  (n  »  9)  that  chose  to  exit,  66%  (n  =  5)  were  low  curious  while  44%  (n  ■  4) 
were  high  curious. 
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Remediation.  Most  children  either  did  not  require  remediation  or  chose 
remediation  when  it  was  offered  the  first  time.  Of  the  71  subjects,  20  subjects 
(28%)  needed  remediation  and  were  asked  if  they  would  like  to  receive  remediation 
after  an  incorrect  response.  Sixteen  of  those  subjects  (80%)  chose  to  be  remediated 
when  it  was  offered.  Four  subjects  (20%)  responded  "No"  to  the  offer  of 
remediation.  Two  of  those  four  were  in  the  Advisement  treatment  and  both  decided 
to  be  remediated  after  receiving  the  advisement, 

Advisement.  When  advisement  was  offered,  it  was  most  often  heeded. 
Unfortunately,  most  children  did  not  get  to  that  point.  Of  those  who  exited  and 
then  received  advisement  to  return  to  the  lesson  (n  =  6),  67%  heeded  the 
advisement  and  returned  to  the  lesson  while  34%  (n  =  3)  chose  to  ignore  the 
advisement. 

As  discussed  under  remediation,  only  two  children  who  chose  not  to  be 
remediated  were  in  the  advisement  treatment.  Both  those  children  heeded  the 
advisement  and  ultimately  received  the  remediation. 

"Stop  and  Look"  and  "Stop  and  Think"  were  the  other  forms  of  advisement  which 
were  built  into  the  lesson.  While  only  the  Advisement  treatment  offered  the 
opportunity  to  "Stop  and  Think;"  both  treatments  offered  the  opportunity  to  "Stop 
and  Look;"  the  advisement  treatment,  however,  included  encouragement  to  "Stop 
and  Look"  as  well.  The  next  section  discusses  the  results  of  the  analysis  of  this 
particular  advisement. 

Effect  of  f*Stop  and  Look"  advisement.  While  second  graders  showed 
little  difference  in  the  mean  number  of  time9  they  used  the  "Stop  and  Look"  option 
between  the  No  Advisement  treatment  (M  =  1.37)  and  the  Advisement  treatment 
(M  =  1.50),  the  difference  between  the  first  graders  in  the  No  Advisement  treatment 
(M  =  .95)  and  first  graders  in  the  Advisement  treatment  (M  =  2.33),  warranted 
closer  examination.  An  independent  means  t-test  was  conducted  with  the  first 
graders  to  determine  whether  there  was  a  significant  difference  between  the  means 
of  the  two  groups  in  use  of  "Stop  and  Look."  The  results  were  significant  at  the  £  < 
.01  level.  It  appears  that  the  first  graders,  then,  actually  heeded  the  advice  to 
"Stop  and  Look"  more  so  than  did  the  second  graders.  Figure  6  illustrates  this 
interaction. 
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Qualitative  Data 

Interview  data  were  collected  from  a  random  selection     25  subjects.  The 
interviews  were  audiotaped.  From  the  25  subjects  interviewed,  twenty  were 
transcribed  for  review;  three  of  the  original  25  were  in  the  middle  curiosity  group 
who  had  been  dropped  from  the  analysis,  and  the  two  remaining  subjects'  audio 
recordings  were  inaudible.  The  20  subjects  accounted  for  28%  of  the  total  sample. 
Through  random  chance,  12  subjects  interviewed  were  high  curious  while  eight 
subjects  were  low  curious,  11  were  males  and  9  were  females,  and  19  had  received 
the  Advisement  lesson  while  1  received  the  No  Advisement  lesson.  There  were  10 
first  graders  and  10  second  graders. 

The  interview  was  conducted  following  the  treatment  and  achievement  test, 
and  prior  to  the  concluding  entertainment  video.  The  purpose  of  the  interviews 
was  to  collect  data  on  how  subjects  felt  about  the  lesson  in  general,  how  subjects 
responded  to  a  learner  control  environment  (since  both  treatments  represented  a 
learner  control  environment),  and  why  subjects  made  certain  decisions.  The 
interview  also  provided  an  opportunity  to  confirm  whether  or  not  the  child  actually 
understood  the  learner  control  options  and  assess  his/her  opinion  about  them.  The 
average  interview  lasted  approximately  4-5  minutes.  In  terms  of  the  results,  the 
most  explanatory  qualitative  information  received  pertained  to  one  of  the  learner 
control  options,  the  opportunity  to  "exit."  It  must  be  noted,  however,  that  almost 
all  who  were  interviewed  had  received  the  Advisement  lesson  which  may  have 
influenced  them  to  stay  in  the  lesson. 

derail  response  to  lesson.  With  the  exception  of  one  "I  don't  know" 
rt  sponse  all  the  subjects  interviewed  had  a  favorable  reaction  to  the  lesson.  The 
most  common  responses  were  "It  was  fun '  and  "I  liked  it"  when  asked  what  they 
thought  about  the  lesson  they  had  just  finished.  When  queried  further,  the  subjects 
varied  in  what  they  considered  "fun"  about  the  lesson.  Some  subjects  enjoyed  the 
technological  aspects  of  the  lesson  such  as  the  opportunity  to  1  touch"  the  computer 
screen  and  make  things  happen,  while  other  subjects  actually  mentioned  the 
content  as  in  "...My  favorite  part  about  it  was  the  big  sculptures  at  the  beginning, 
and  the  artists  were  neat  and  the  different  kinds  ot  portraits  and  stuff." 

Exiting.  Most  children  did  not  exit  the  lesson  as  indicated  by  the  lesson 
Jog  completed  by  the  administrators.  This  undoubtedly  contributed  to  the  lack  of 
treatment  effects.  Had  more  children  exited,  it  would  have  been  possible  to 
determine  the  effect  of  advisement  on  their  decisions  to  re-enter  the  lesson  or  not. 
The  interviews  helped  to  explain  why  more  children  never  got  to  this  advisement. 
Typically,  the  children  responded  that  they  did  not  exit  because  they  either  were 
enjoying  their  experience  or  were  interested  in  what  else  would  be  coming  up  in  the 
lesson.  Again,  the  children  interviewed  were  mostly  in  the  Advisement  lesson  and 
may  have  felt  more  comfortable  about  staying  in  the  lesson  as  a  result.  Only  13% 
of  the  71  subjects  chose  to  exit.  The  following  is  a  typical  exchange  of  those 
interviewed. 

Interviewer 
Remember  that  big  red  button? 

Child 

Yeah.  That  wan  the  exit. 
Interviewer 

Thiit'H  correct.  You  never  touched  that  exit  button.  Why  didn't  you  ever  press  the  exit  button?   ■*  i 

Child 

1  don't  know.  I  just  wanted  to  see  what  else  was  there. 
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The  above  excerpt  was  taken  from  an  interview  with  a  high  curious  child. 
The  responses  from  low  curious  children  were  similar.  The  following  excerpt  is 
from  an  interview  with  a  low  curious  child. 

Interviewer 

I  noticed  that  you  didn't  use  the  big  red  button.  Do  you  remember  that  red  button? 

Child 
Yeah. 

Interviewer 
What  did  that  rtd  button  stand  for? 

Child 

Um,  if  you  didn't  want  to  do  that  any/nore. 

Interviewer 

That's  right.  You  could  what? 

Child 

Stop. 

Interviewer 

Stop.  Ymi  could  JuKt  gut  right  out  of  thu  lesson.  But,  you  never  pushed  that  red  button.  How  come? 

Child 
Because  I  liked  it. 

A  particularly  enthusiastic  second  grader  (high  curious)  had  this  exchange 
with  the  interviewer: 

Interviewer 

Did  you  ever  exit,  touch  the  red  button? 

Child 

No. 

Interviewer 

How  come? 

Child 

BecaUHe  I  thought  it  was  really  interesting  just  looking  at  what  I  picked.  I  didn't  even  pay  any  allontttm 
[to  the  red  button]  it  was  so  cool.  I  just  liked  what  I  picked. 

Learner  Control,  Regardless  of  whether  a  child  was  in  the  Advisement  or 
the  No  Advisement  treatment,  they  were  functioning  in  a  learner  control 
environment  in  which  they  had  to  make  decisions  about  their  own  learning.  The 
children  were  asked  about  what  it  felt  like  to  make  their  own  decisions  as  well  os 
asked  what  kind  of  decisions  they  made.  It  should  be  noted  once  again  that  more 
high  curious  subjects  were  interviewed  than  low  curious  subjects  so  it  is  difficult  to 
make  general  statements  regarding  the  low  curious  subjects. 
All  high  curious  subjects  and  five  of  the  seven  low  curious  children  enjoyed  heintf 
able  to  make  their  own  decisions  in  the  lesson,  When  queried,  most  of  them 
remembered  at  least  some  of  the  actual  decisions  they  made  such  as  the  order  in 
which  they  selected  the  content  or  whether  they  used  options  such  as  "Stop  and 
Look/'  The  first  exchange  takes  place  with  a  low  curious  female  who  had  received 
the  Advisement  treatment 
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Interviewer 

In  a  lesson  like  this,  you  get  to  make  lots  of  decisions.  Like  what  you  will  learn  about,  whether  to  exit, 
whether  to  practice  more  if  you  get  a  wrong  answer,  and  things  like  that.  In  this  lesson,  you're  the  boss. 
What  does  it  feel  like  being  the  boss?  [making  decisions  in  the  lesson] 

Child 

It  feels  good.  Like  you're  controlling  yourself. 

Interviewer 

Is  lhat  a  good  feeling  being  able  to  control  yourself? 

Child 
Yeah. 

Interviewer 

Do  you  get  to  do  that  much  in  your  life  at  your  age? 

Child 

Well. ..[pause]... not  always.  Like  if  I  go  in  my  room,  I  have  a  sign  on  my  door  that  tells  people  that  they 
have  to  knock.  And  then  I  can  control  myself  when  I'm  in  the  room,  but,  like  in  school  I  have  to  do 

certain  things  and  stuff  like  that. 

Interviewer 

And  how  is  this  [lesson]  different? 

Child 

Well,  you  can  choose  the  things  that  you  want  to  do  and  that  you  would  like  to  see... 

The  reference  to  "control1  was  used  by  this  high  curious  male  subject  as 

well... 

Interviewer 
...What  was  that  like...to  be  the  boss? 

Child 

It  was  like  I  was  controlling  the  whole  world  or  the  country. 

Interviewer 
And  how  did  that  feel  to  you? 

Child 

It  felt  like  I  was  Georgo  Washington  or  the  Governor  of  the  World! 

The  low  curious  female  in  the  following  excerpt  did  not  like  the  idea  of 
exploring  information  on  her  own. 

Interviewer 

Do  yon  like  people  telling  you  about  the  information,  or  do  you  like  to  investigate  and  find  information 

out  yourself? 

Child 

I  like  people  tailing  me,,, 

The  qualitative  data  although  sampling  only  28%  of  the  subjects,  helped  to 
illuminate  iBSuei  such  as  the  lack  of  treatment  differences. 

Overall,  the  high  curious  children  in  this  study  responded  to  an  interactive 
learner  control  environment  more  effectively  than  the  low  curious  children 
regardless  of  which  lesson  they  received  as  indicated  by  performance  on  an 
achievement  test  related  to  the  lesson  content 
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DISCUSSION 


The  emergence  of  interactive  learning  technologies  provides  educators, 
designers  and  developers  with  an  opportunity  and  a  challenge  to  meet  the 
individual  needs  of  different  types  of  learners.  To  achieve  desired  instructional 
outcomes,  the  educator  or  designer  must  consider  various  instructional  variables  in 
order  to  prescribe  an  appropriate  instructional  method  (Reigeluth,  1983).  For  this 
study,  the  instructional  variable  explored  was  learner  curiosity.  The  method 
explored  was  learner  control.  Learner  control  as  it  relates  to  individual  differences 
is  an  important  issue.  While  research  exists  in  the  separate  areas  of  learner 
control  and  curiosity,  there  is  little  known  about  the  relationship  between  these 
variables  and  what  effect  they  have  on  learning. 

This  study  built  upon  the  Arnone  and  Grabowski  (1992)  research  on  the 
effects  of  variations  in  learner  control  on  young  children's  learning.  That  study 
showed  that  young  learners  demonstrated  significantly  better  achievement  in  a 
learner  control  lesson  which  incorporated  advisement  than  one  in  which  the  learner 
received  no  advisement. 

The  Arnone  and  Grabowski  (1992)  study  did  not  look  at  individual 
differences  in  trait  curiosity.  A  learner  control  environment  might  be  experienced 
differently  by  individuals  who  vary  in  trait  curiosity.  The  curiosity  research 
suggests  that  individuals  have  different  tolerance  levels  for  arousal  potential  (Day 
&  Berlyne,  1971).  Varying  degrees  of  learner  control,  then,  may  present  different 
levels  of  arousal  potential  depending  on  the  individual.  Examination  of  the  two 
separate  literatures  provided  a  basis  for  a  conceptualization  of  a  theoretical 
framework  for  the  hypotheses  presented  in  the  study. 

Curiosity 

Curiosity  and  Learner  Control 

High  curious  subjects  were  predicted  to  perform  better  than  low  curious 
subjects  in  either  learner  control  environment,  High  curious  subjects,  having  a 
higher  tolerance  for  uncertainty  and  the  unfamiliar,  should  be  more  efficient  in  a 
learner  control  environment.  Low  curious  subjects,  on  the  other  hand,  may  feel 
overstimulated  or  become  anxious  or  disinterested  in  such  a  learning  situation. 
The  results  of  this  study  supported  that  prediction  (]>  <  .01).  High  curious  children 
achieved  significantly  higher  scores  on  an  achievement  test  than  low  curious 
children  after  being  exposed  to  a  learner  control  lesson,  regardless  of  treatment. 
This  would  support  the  theoretical  argument  that  individuals  have  different 
optimal  levels  of  arousal.  What  is  the  appropriate  amount  of  stimulation  for  one 
person  may  not  be  enough  or  may  be  too  much  for  another. 

In  addition  to  the  art  education  content,  a  certain  amount  of  arousal 
inducing  stimuli  were  present  in  this  lesson  by  virtue  of  its  being  a  learner  control 
environment.  Such  stimuli  included  use  of  touch  screen,  choice  of  menu  options, 
and  use  of  other  learner  control  options  such  as  pacing,  remediation,  exiting,  "Stop 
and  Look "  and,  in  the  Advisement  lesson,  the  length  of  time  to  spend  in  the  "Stop 
and  Think"  mode.  Taken  together,  the  stimuli  described  above  could  be  considered 
rather  high  in  eollative  variability,  a  construct  used  by  Berlyne  (1960.  Collative 
variability  refers  to  how  complex  a  stimulus  is  and  how  easy  or  difficult  it  is  for  the 
individual  to  collate,  or  compare  with  stimuli  with  which  he/she  is  already  familiar. 
While  collative  properties  have  the  potential  of  increasing  arousal  level  and 
inducing  curiosity,  Day  and  Berlyne  (1971)  state  that  different  individuals  will  vary 
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in  their  preference  and  tolerance  for  arousal  potential.  In  this  lesson,  the  high 
curious  subjects  were  able  to  function  efficiently,  both  assimilating  the  content  and 
collating  the  various  stimuli  presented  via  the  learner  control  format. 

On  the  other  hand,  the  low  curious  subjects  may  have  spent  more  of  their 
effort  in  attempting  to  collate  the  various  stimuli  of  a  learner  control  lesson  which 
limited  their  efficiency  in  terms  of  assimilating  the  necessary  content.  This  was 
reflected  in  lower  achievement  scores. 

The  re  llts  of  this  study  provide  some  evidence  to  demonstrate  that 
curiosity  is  an  important  factor  in  either  learner  control  environment.  These  high 
curious  students  functioned  better  in  this  situation  than  the  low  curious  students. 
However>  it  is  not  known  how  these  high  curious  students  would  have  performed  in 
a  program  control  environment.  The  high  curious  subjects  hypothetically  would 
perform  less  well  in  a  program  control  environment  because  the  amount  of  stimuli 
would  be  less  than  optimal  for  them.  For  the  low  curious  students,  program  control 
may  have  the  appropriate  amount  of  stimuli  for  them,  resulting  in  more  efficient 
learning.  In  the  present  study,  it  is  likely  that  the  low  curious  subjects  experienced 
both  treatments  as  at  least  somewhat  high  in  degree  of  uncertainty.  Further 
research  is  needed  to  explore  this  as  well  as  research  to  investigate  the  effect  of  low 
uncertainty  (e.g.,  program  control)  on  learning  in  low  curious  subjects  since  this 
study  did  not  include  program  control.  Fry  (1972),  for  example,  found  that  high 
aptitude/highly  inquisitive  subjects  learned  significantly  more  in  what  that  author 
called  a  high  degree  of  student  control  (what  is  referred  to  as  learner  control  in  this 
study)  while  high-aptitude/low-in-inquisitiveness  subjects  performed  significantly 
better  given  a  low  degree  of  student  control. 

Alternative  Explanation;  Why  Subjects  Did  Not  "Exit" 

Earlier,  several  explanations  were  offered  to  explain  why  more  subjects  did 
not  exit  the  lesson.  (By  not  exiting,  they  never  received  the  advisement  pertaining 
to  the  exit  decision  which  lessened  the  effect  of  the  Advisement  treatment.)  One 
suggestion  was  that  most  children  did  not  leave  because  they  were  engaged  in  the 
lesson  and  simply  wanted  to  remain  in  the  lesson.  The  randomly  selected 
interviews,  representing  28%  of  the  sample,  substantiated  this.  (It  should  be 
repeated  that  only  eight  low  curious  subjects  as  opposed  to  12  high  curious  subjects 
were  interviewed,  and  most  subjects  interviewed  had  received  the  advisement 
lesson.)  Exiting  would  have  provided  the  low  curious  subjects  a  means  to  limit 
their  stimulation  by  actually  withdrawing  from  the  source  of  it,  the  learner  control 
lesson  itself.  Yet,  85%  (n  s  31)  of  the  low  curious  subjects  remained  in  the  lesson. 
The  following  paragraphs  offer  an  alternative  explanation  for  why  more  low  curious 
subjects  did  not  exit. 

Since  the  subjects  in  this  study  were  young  and  inexperienced,  exiting  the 
lesson  might  have  represented  to  them  a  choice  that  was  even  more  unfamiliar  to 
them  than  the  lesson  itself  which  provided  them  with  some  regularity  of  format  and 
presentation.  In  other  words,  the  longer  they  stayed  in  the  lesson,  the  less  likely  it 
became  that  they  would  exit  since  some  of  the  uncertainty  of  a  learner  control 
environment  diminished  simply  as  a  result  of  staying  in  the  lesson.  For  example, 
as  more  time  was  spent  in  the  lesson,  they  became  familiar  with  the  sound  of  the 
narrator's  voice,  and  with  the  format  of  practice  items  and  with  touching  the 
screen.  The  lesson  itself  became  the  more  familiar  scheme  while  the  red  exit 
button  may  have  embodied  an  object  of  uncertainty.  Not  yet  having  touched  it,  it 
was  something  that  might  have  represented  some  cognitive  conflict:  what  really 
would  happen  if  the  red  button  were  touched?-perhaps  the  screen  would  become 
black,  perhaps  it  would  mean  an  immediate  return  to  the  classroom,  and  so  on. 
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Even  though  they  had  been  told  in  the  pre-lesson  warm-up  that  the  lesson  would 
end  if  they  touched  the  red  button,  if  they  had  never  actually  done  it  in  the  lesson, 
it  would  represent  an  unexplored  stimulus. 

If  the  above  explanation  is  true,  it  would  follow  that  an  individual  who 
prefers  a  lower  level  of  arousal  would  choose  to  ignore  this  thus  far  unexplored  and 
therefore  unfamiliar  button,  a  button  which  posed  an  additional  source  of  collative 
variability.  This  explanation  can  be  related  to  Day  and  Berlyne's  statement  about 
collative  properties  and  response  conflict: 

Collative  properties  act  by  inducing  a  state  of  response  conflict,  i.e.,  uncertainty  about  the 
nature  of  the  stimulus  and  thus  a  conflict  among  the  various  response  tendencies  toward  it. 
(1971,  p.  313) 

It  is  possible,  then,  that  uncertainty  about  the  nature  of  the  exit  button 
induced  a  response  in  these  learners  which  may  have  been  resolved  by  avoidance. 

Effect  of  Advisement  on  Low  Curious  Subjects 

There  was  a  difference  of  1.4  points  between  treatments  in  favor  of  the 
Advisement  treatment  with  low  curious  subjects.  The  minimum  significant 
difference  was  3.17  and  therefore  this  difference  was  not  statistically  significant  at 
the  .05  level.  The  treatments  were  equalized  because  students  often  did  not  receive 
the  advisements  related  to  exiting  and  remediation.  They  did  not  receive  the  exit 
advisement  because  they  did  not  press  the  exit  button;  they  did  not  receive  the 
remediation  advice  because  they  generally  took  the  remediation  when  offered, 

The  advice  to  "Stop  and  Look"  in  the  Advisement  treatment  did  make  a 
difference,  however,  in  the  number  of  times  that  the  option  was  used  by  low  curious 
subjects.  Low  curious  subjects  responded  to  the  advisement  to  "Stop  and  Look" 
significantly  more  times  in  the  advisement  treatment  (M  =  2.06)  than  did  the  low 
curious  subjects  in  the  No  Advisement  treatment  (M  =  1.16).  The  minimum 
significant  difference  was  0.826;  the  difference  was  significant  at  the  £  <  .05  level. 
Th;s  seems  to  indicate  that  this  advisement  in  the  Advisement  treatment  did 
reduce  some  of  the  uncertainty  associated  with  that  stimulus  as  compared  to  the 
No  Advisement  treatment.  What  this  suggests  overall  is  that  such  advisements  did 
make  a  difference  for  the  low  curious  subjects  in  that  they  responded  to  them  (as 
evidenced  by  the  use  of  them),  but  on  its  own  it  was  not  enough  to  make  a 
statistical  difference  in  scores  between  the  treatments.  It  might  further  be 
suggested  that  low  carious  subjects  will  respond  to  advisement  if  advisement  is 
received.  In  the  case  of  exiting  and  remediation,  such  advisement  was  not  received 
for  mo9t  of  the  subjects. 

Perhaps,  more  such  strategies  along  the  lines  of  "Stop  and  Look"  and  MStop 
and  Think"  embedded  in  the  lesson  would  have  cumulatively  contributed  to  making 
advisement  more  beneficial  to  low  curious  students. 

Treatment 

Diminished  Power  of  Treatment 

One  explanation  for  the  lack  of  a  main  effect  for  treatment  was  that  the 
differences  between  the  treatments  were  diminished  because  most  subjects  in  the 
advisement  treatment  did  not  get  the  full  benefit  of  each  aspect  of  advisement. 

Remediation.  Most  of  the  subjects  in  the  Advisement  treatment  took 
remediation  the  first  time  it  was  offered  and  thus  never  had  advisement  pertaining 
to  remediation.  The  subjects  in  the  No  advisement  treatment  also  took  the 
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remediation  when  offered.  This  may  have  contributed  to  the  lack  of  differences 
between  the  groups  which  would  have  been  attributed  to  receiving  or  not  receiving 
advisement  on  remediation. 

Exiting.  Another  possible  reason  for  no  main  effect  for  treatment  was  that 
very  few  children  exited  the  program.  In  the  Advisement  treatment,  if  a  child 
exited,  he/she  would  have  received  a  prompt  like  this:  "Are  you  sure  you  want  to 
leave  now?  There's  lots  more  for  you  to  discover.  This  next  section  is  very 
interesting."  The  advisement,  then,  gave  the  child  the  opportunity  to  reconsider 
his/her  decision  and  re-enter  the  lesson.  To  receive  the  advisement,  however,  the 
child  must  first  have  pressed  the  exit  button.  Had  more  children  in  both 
treatments  exited,  it  would  have  been  possible  to  sense  the  effect  of  this  advisement 
on  performance  in  the  achievement  test.  That  is,  those  that  heeded  the  advisement 
and  re-entered  the  lesson  would  be  expected  to  demonstrate  higher  achievement 
scores  than  those  who  did  not  heed  the  advisement  ^r  did  not  receive  the 
advisement,  thus,  missing  out  on  some  of  the  content  presentation  and  spending 
less  time  on  task. 

Questions  to  be  resolved  include  whether  the  subjects  were  too  afraid  to  exit 
(e.g.,  they  may  have  had  the  expectation  that  they  were  "supposed"  to  finish)  or 
whether  they  were  too  engaged  in  the  lesson  to  exit.  Because  of  the  careful 
procedures  followed  in  the  pre-lesson  warm-up,  it  seems  unlikely  that  the  reason 
children  did  not  press  the  red  exit  button  was  because  of  perceived  expectations 
that  the  lesson  had  to  be  completed.  Each  administrator  followed  a  three-step 
procedure  related  to  the  exit  button  as  follows:  1)  introduce  the  child  to  the  red  exit 
button  and  explain  its  function,  2)  have  the  child  relate  his/her  understanding  of 
the  button's  function  (to  confirm  that  child  understood  the  directions),  and  3)  tell 
the  child  that  it  is  permissible  to  use  the  exit  button  anytime  he/she  wants  to  end 
the  lesson.  The  last  part  of  the  procedure  was  very  important;  it  was  designed  to 
alleviate  the  child's  potential  fear  of  expectations  on  the  part  of  the  administrator 
that  the  child  had  to  finish  the  lesson.  It  is  equally  important  to  note,  however, 
that  while  the  administrator  used  the  three  procedures  described  above  to  assure 
that  each  child  understood  the  meaning  and  function  of  the  red  exit  button,  the 
child  would  not  have  the  opportunity  to  actually  exit  until  he/she  was  in  the  actual 
lesson. 

Some  children  may  not  have  exited  because  it  would  have  meant  returning 
to  their  classroom.  Working  in  the  CBIV  lesson  may  have  seemed  more  appealing 
to  them  than  the  alternative.  Many  children  had  not  experienced  interactive  video 
prior  to  the  study,  and  so  the  novelty  effect  may  also  have  contributed  to  keeping 
them  in  the  lesson. 

Another  possible  reason  that  most  children  did  not  exit  is  that  they  were  so 
engaged  in  the  lesson  that  they  wanted  to  remain  in  it.  The  interviews 
representing  28%  of  the  children  helped  to  corroborate  this  explanation.  A  caution, 
discussed  earlier,  remains  for  this  explanation  since  most  of  those  who  were 
interviewed  had  received  the  Advisement  lesson;  therefore,  those  receiving  the  No 
Advisement  lesson  were  under- represented  in  the  interviews,  Part  of  the  interview 
was  devoted  to  exploring  reasons  for  decisions  made  by  the  child.  The  exiting 
decision  was  usually  discussed  as  in  "I  noticed  that  you  did  not  use  the  red  button 
to  leave  the  lesson.  Why  not?"  The  most  typical  response  was  "I  didn't  want  to.  It 
was  fun."  It  may  be  that  children  were  engaged  by  the  changing  elements  in  the 
lesson  (e.g.,  new  video  segments,  new  content,  etc.)  and  that  they  attended  more  to 
these  changing  elements  than  to  the  static  exit  button.  The  child  quoted  in 
Chapter  4  as  saying  "...I  didn't  even  pay  any  attention  [to  the  red  button]  it  [the 
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lesson]  was  so  cool,"  suggests  this  explanation,  Berlyne  (1951)  states: 

A  recently  changed  stimulus  is  more  likely  to  be  responded  to  than  one  which  has  remained 
unchanged  and  has  been  responded  to  for  some  time.  (p.  277) 

While  the  exit  button  would  have  represented  a  stimulus  which  remained 
"unchanged'  as  the  lesson  progressed,  it  would  not  have  represented  a  stimulus 
which  had  been  "responded  to  for  some  time"  (since  most  children  did  not  ever 
respond  to  the  exit  button  by  pressing  it).  It  would  thus  appear  that  there  may  be 
some  other  alternative  explanation  worth  investigating.  Such  an  explanation  deals 
with  level  of  curiosity  which  is  discussed  later  in  this  chapter. 

In  sum,  not  exiting  may  have  had  the  effect  of  diminishing  the  strength  of 
the  treatments  making  them  less  sensitive  to  differences  due  to  advisement  about 
exiting  decisions. 

Power  of  Treatments  Left  to  Remaining  Advisement  Strategies 

Effectively,  the  Advisement  treatment  was  differentiated  from  the  No 
Advisement  treatment  by  the  two  remaining  embedded  advisement  strategies:  1) 
"Stop  and  Look"  and  2)  "Stop  and  Think."  Although  both  treatments  offered  the 
opportunity  to  "Stop  and  Look"  at  an  image  whenever  a  particular  icon  was 
present,  subjects  in  the  Advisement  treatment  were  also  encouraged  to  use  the 
"Stop  and  Look"  option  to  explore  the  images  more  closely.  Subjects  in  the 
Advisement  treatment  were  also  encouraged  to  "Stop  and  Think"  periodically  as  in 
"What  do  you  suppose  you'll  find  when  you  go  in  [the  museum  I?  'Stop  and  Think1 
about  it.  Then  touch  the  screen  when  you  are  ready  to  go  on." 

Without  the  contribution  of  the  remediation  and  exiting  advisements  in  the 
Advisement  treatment,  the  strength  of  the  treatment  fell  to  the  above  remaining 
advisement  strategies.  It  seems  likely  that  these  strategies  were  not  enough  on 
their  own  to  sense  the  effect  of  advisement  versus  no  advisement  as  a  main  effect. 
It  may  be  that  the  two  treatments  were  equalized,  in  effect,  by  the  subjects' 
enjoyment  of  the  lesson.  It  might  also  have  made  a  difference  overall  had  more 
embedded  advisement  strategies  been  incorporated  in  the  treatment. 

Explanation  of  Grade  by  Treatment  Interaction 

Although  the  power  of  the  treatment  may  have  been  diminished  overall,  the 
first  grade  subjects  performed  significantly  better  with  Advisement  than  with  No 
Advisement.  The  opposite  was  true  and  significant  for  second  graders.  A  possible 
explanation  for  these  results  pertains  to  the  remaining  advisement  stiategies  and 
their  effect  on  the  younger  subjects. 

Interpretation  of  results  with  first  graders/  Examination  of  the  in- 
treatment  data  showed  that  the  first  grade  subjects  heeded  the  advisements  to 
"Stop  and  Think"  and  "Stop  and  Look"  more  so  than  the  older  subjects,  Evidence 
supporting  this  supposition  was  collected  for  one  of  these  advisements.  Although 
no  data  wero  collected  on  the  time  spent  in  the  "Stop  and  Think"  mode,  data  were 
collected  on  the  number  of  times  a  subject  used  the  "Stop  and  Look"  learner  control 
option.  Grade  1  children  in  the  Advisement  treatment  did  use  the  "Stop  and  Look" 
option  more  times  (M  «  2.33)  than  did  grade  1  children  in  the  No  Advisement 
treatment  (M  *  0.95).  The  difference  was  significant  at  the  £  <  .05  level  using  the 
Nnwtnnn-Kouls  procedure.  More  importantly,  first  graders  in  the  Advisement 
treatment  also  used  the  "Stop  and  Look"  learner  control  option  significantly  more 
tlmi'8  than  their  second  grade  counterparts  in  the  same  treatment  (M  =  1.50),  £  < 
.05.  These  in-treatment  data  suggest  that  the  first  graders  actually  heeded  the 
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advisements  to  a  greater  degree  than  the  older  subjects  and  may  be  partly 
responsible  for  the  better  performance  of  first  graders  over  second  graders  in  the 
Advisement  treatment. 

Why  would  first  graders  heed  advice  more  than  the  second  graders?  One 
possible  explanation  is  related  to  the  newness  of  the  academic  experience  for  these 
younger  subjects.  The  first  graders  were  less  experienced  and  may  have  perceived 
the  voice  of  the  narrator  as  a  reassuring  adult  authority  figure  offering 
encouragement  to  explore  images  carefully  and  to  "Stop  and  Think."  They  may  also 
have  required  more  guidance  as  their  experience  with  computers  was  minimal 
compared  with  the  second  graders. 

Interpretation  of  results  with  second  graders.  With  second  graders, 
the  explanation  of  the  grade  by  treatment  interaction  is  more  nebulous.  There  was 
no  significant  difference  between  treatments  in  the  use  of  the  "Stop  and  Look" 
learner  control  option  with  second  graders.  The  second  graders  actually  performed 
better  in  the  No  Advisement  treatment  than  in  the  Advisement  treatment.  Why 
did  second  graders  not  heed  the  advisements  as  much  as  the  younger  subjects? 
One  possible  explanation  is  that  computer-based  instruction  was  not  as  new  to 
these  subjects;  they  perhaps  did  not  sense  the  need  for  encouragement  from  the 
narrator  as  much  as  the  younger  subjects.  Additionally,  second  graders  had  more 
experience  with  computers  in  the  classroom  and  may  have  felt  that  they  were  more 
experienced  users  and  did  not  need  to  follow  the  advice. 

What  the  results  seem  to  indicate  is  that  when  advisement  is  heeded,  as  in 
the  case  of  the  first  graders,  it  does  make  a  difference.  When  it  is  not  heeded,  as  in 
the  case  of  the  second  graders,  it  seems  likely  that  learners  can  do  just  as  well  or 
better  in  a  learner  control  lesson  without  advisement.  Furthermore,  it  seems  likely 
that  younger  subjects  will  heed  advisement  more  than  older  subjects. 

Comparison  to  Bridge  Study 

This  study  was  at  odds  with  the  bridge  study  in  that  no  main  effects  for 
treatment  were  found.  However,  the  overall  significant  differences  for  the  bridge 
study  were  very  closely  replicated  in  the  first  grade  population  of  the  present  study. 
That  is,  the  mean  score  for  subjects  in  the  learner  control  with  advisement 
treatment  in  the  bridge  study  was  16.08,  while  the  mean  score  for  first  graders  in 
the  present  study  was  16.39.  The  mean  score  for  the  learner  control  without 
advisement  treatment  in  the  bridge  study  was  13.35  while  the  respective  score  for 
first  graders  in  the  present  study  was  12.33. 

The  anomaly  seems  to  have  occurred  with  the  second  graders  who 
performed  contrary  to  expectations.  The  bridge  study  did  not  isolate  the  grade 
variable  so  comparisons  across  studies  by  grade  are  impossible.  However,  upon 
close  scrutinization  of  the  data  for  the  present  study,  an  interesting  pattern 
emerges.  There  were  more  high  curious  subjects  in  the  second  grade  (n  =  20)  than 
in  the  first  grade  (n  =  12).  There  were  also  more  low  curious  subjects  in  the  first 
grade  In  =  23)  than  in  the  second  grade  (n  =  12).  Furthermore,  there  were  more 
high  curious  second  graders  in  the  No  Advisement  treatment  (n  =  10)  than  there 
were  low  curious  second  grade  subjects  in  the  No  Advisement  treatment  (n  =  6).  It 
is  possible,  then,  that  the  high  curious  subjects  who  predominated  in  the  No 
Advisement  treatment  in  the  second  grade  contributed  to  the  unexpectedly  high 
scores  in  the  No  Advisement  treatment  with  second  graders.  Since  high  curious 
subjects  were  expected  to  experience  greater  ease  in  handling  the  uncertainty  of  the 
No  Advisement  treatment,  this  is  one  possible  explanation  for  anomaly  which 
occurred. 
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Implications  for  Instructional  Design 

This  study  provided  empirical  support  for  the  hypothesis  that  high  curious 
children  learn  better  in  two  types  of  learner  control  lessons  than  low  curious 
children.  Such  information  has  implications  for  the  design  of  instruction  whether 
the  delivery  of  that  instruction  is  in  a  classroom  or  via  interactive  technologies. 
Based  on  the  findings,  it  is  recommended  that  educators  and  designers  acquire 
knowledge  about  the  individual  differences  in  curiosity  level  among  students  and 
determine  appropriate  methods  and  strategies  to  maximize  learning  in  each 
individual  student.    If,  for  example,  it  is  known  that  a  particular  chile  is  very  low 
curious,  then  the  child  may  be  overwhelmed  in  any  learner  control  environment  at 
this  point.  A  high  amount  of  guidance  in  the  way  of  instructor  (classroom)  or 
program  (CBIV)  control  of  the  lesson  would  be  recommended.  In  other  cases,  where 
the  child  is  mid-to-low  in  curiosity,  a  learner  control  lesson  with  many  embedded 
strategies  to  reduce  uncertainty  and  bring  the  lesson  in  line  with  the  child's 
optimal  level  of  arousal  potential  would  be  appropriate.  A  high  curious  child 
should  feel  comfortable  in  an  environment  with  moderate  to  high  amounts  of 
learner  control. 

Based  on  the  superior  performance  of  the  first  grade  subjects  in  the 
Advisement  treatment  in  this  study,  and  the  results  in  favor  of  advisement  in  the 
Arnone  and  Grabowski  (1992)  study,  advisement  is  recommended  in  a  learner 
control  lesson  designed  for  young  learners,  What  seems  to  be  even  more  important, 
however,  is  that  advisement  must  be  heeded  in  order  to  be  successful.  Strategies, 
therefore,  must  be  employed  which  also  encourage  students  to  heed  advisement. 

Suggestions  for  Future  Research 

Research  needs  to  be  conducted  which  addresses  some  of  the  limitations  of  the 
present  study,  Since  the  power  of  the  treatments  was  diminished  by  the  fact  that 
subjects  did  not  use  all  the  advisements,  a  future  study  could  strengthen  the 
respective  treatments  by  including  more  learner  control  options  and  more 
opportunities  for  advisement.  In  this  way,  the  loss  of  two  advisements  as  was  the 
case  in  the  present  study  (i.e.,  exiting  and  remediation)  would  not  be  as  detrimental 
to  sensing  the  overall  effect  of  the  treatments. 

Perhaps,  research  needs  to  be  undertaken  which  explores  who  takes 
advisement  and  who  does  not,  as  well  as  what  strategies  are  most  effective  in 
inducing  students  to  heed  advisement.  Future  studies  should  also  include  program 
control,  Although  this  study  predicted  that  low  curious  subjects  would  perform 
better  in  an  environment  where  there  was  less  collative  variability,  program  control 
was  not  included.  Program  control  would  take  the  learner  along  a  predetermined 
path  and  might  substantially  reduce  the  uncertainty  of  the  learning  environment. 
A  study  which  includes  program  control  in  addition  to  variations  in  learner  control 
could  be  used  to  investigate  the  effect  on  learners  of  the  full  range  of  uncertainty  in 
learner  control  as  depicted  in  Figure  2. 

More  psychometric  studies  need  to  be  conducted  to  collect  data  on  the 
curiosity  instruments,  especially  the  Specific  Curiosity  Scale  for  Primary  Grades 
(SCSPG)  so  that  the  instruments  can  continue  to  be  improved  in  terms  of  their 
validity  and  reliability  in  measuring  curiosity  in  young  subjects. 

Is  it  possible  that,  with  repeated  exposure  to  a  learner  control  environment, 
the  uncertainty  associated  with  this  type  of  learning  environment  would  be 
reduced?  It  would  seem  that  low  curious  subjects  might  then  improve  their 
performance  as  they  became  more  familiar  with  the  types  of  stimuii  associated  with 
a  learner  control  lesson,  These  stimuli  would  include  the  learner  control  options  in 
addition  to  the  specific  content  presented,  To  research  this  question,  a  study  could 


12,: 


no 


21) 

he  deilgued  whirl*  collects  data  over  time  on  the  use  of  learner  control  lessons  and 
(In;  ihangKS  in  achievement  with  increased  exposure,  A  variation  of  thie  suggestion 
would  la*  studies  which,  using  program  control  initially,  gradually  reduce  the 
amount  or  program  control  while  increasing  learner  control  10  that  learner!  build 
up  to  tolerating  higher  levels  of  uncertainty. 

Future  studies  should  also  be  planned  which  explore  the  potential  for 
increasing  trait  curiosity  level  over  time.  Berlyne  (1960)  discussed  several  studies 
in  which  Rtimuli  previously  associated  with  supraoptimal  arousal  for  a  child  became 
optimal  a,s  the  child's  experience  with  the  situation  increased.  A  learner  control 
study  could,  for  example,  be  undertaken  which  examines  the  effectiveness  of 
particular  instructional  design  strategies  designed  to  enhance  state  curiosity  (e.g., 
encouragement  to  explore  with  guidance  provided,  use  of  questions  to  arouse 
interest,  encouragement  of  question-asking  behavior,  provision  of  environment  in 
which  experimentation  can  occur,  etc.)  and  determine  whether  trait  curiosity  can  be 
enhanced  over  time  through  repeated  experiences  in  which  state  curiosity  is 
increased    In  conjunction  with  the  same  study  or  a  subsequent  study,  changes  in 
trait  curiosity  should  be  correlated  with  changes  in  achievement  to  determine  if 
(here  is  a  significant  positive  relationship  between  these  two  variables  as  would  be 
predicted. 

Finally,  the  uhc  of  physiologic  measures  may  provide  physical  evidence  of 
an  individual's  curiosity  as  measured  by  arousal  during  a  CBIV  lesson.  Clariana 
1 1900)  found  OtSH  to  be  correlated  highly  with  measures  which  are  of  interest  to 
researchers  in  the  field  of  instructional  design.  For  example,  Raskin  (1973)  used 
USK  hb  a  measure  nf  attention  Other  researchers  have  found  strong  correlations 
between  OSH  and  attitude  self-reports  and  preference  (e.g.,  Schwartz  &  Shapiro, 
1973). 


Ill  ; 


liKFKIUSNl!KH 


Ahramnt  A.  (1986).  Effectiveness  of  interactive  video  in  teaching  basic  photography 
skills.  Paper  presented  at  the  Annual  Convention  of  (ho  Association  for 
Educational  Communications  anti  Technology  (Las  Vegan,  NV,  January  Hi  21), 


Arnone  M.t  and  Grabowski,  B.  (1992).  Effects  on  achievement  and  curiosity  of 
variations  in  learner  control  in  an  interactive  video  lesson.  Educational  Technology 
Research  and  Development  V40,  L 

Arnone  M  ,  atid  Hciretta,  M.  (1991).  Specific  curiosity  scale  for  primary  grades. 
Unpublished  experimental  measure. 

Balson  P.,  Manning  D.,  Ehner  II,  &  Brooks,  F.  <  1985)  Instructor-controlled  versus 
student-controlled  training  in  a  vldeodiBc-based  paramedical  program.  Journal  of 


Berlyne,  I).  K.  (19(i0).  Conflict.  Arousal,  and  Curiosity  New  York:  McUraw  Mill 
Book  Company. 

Bosco,  J.  ami  Wagner,  J  ( 19HH).  A  comparison  of  I  he  effectiveness  of  inleractive 
laser  disc  and  classroom  video  (ape  for  safety  instruction  of  General  Motors 
workers,  Educational  Technology,  V28,  &  15  22 

Cecil,  L.,  Gray,  M  ,  Thornhtirg  K  ,  and  Ispa  J  <  1985)  Curiosity  exploration  play 
creativity:  The  early  childhood  mosaic  Early  Childhood  Development  and  Care, 
VI9,  199-217. 

(Mariana,  It.  (1990),  Gender  ami  ability  attentional  differences  while  watching  an 
educational  television  program.  ERIC  Document  #  EIKMi.'lHi  12  p 

Dalton,  D.  W.  (1986).  The  etricacy  of  computer-assisted  video  instruction  on  rule 
learning  and  attitudes.  Journal  of  Computer-Baaed  Instruction,  V13.  122  125. 

Day,  II  (liifiH).  Role  of  specific  curiosity  in  school  achievement  Journal  of 
Educational  Psychology,  V59,  i,  37-4.1 

Day,  II  I  ,  and  Berlyne  D.  E  (1971).  Intrinsic  motivation.  In  11.  S  Lesser  (Ed.), 
Psychology  and  Educational  Practice  Glenview,  Illinois.  Scott,  Foresman  and 
Company,  294-1135. 

Day,  II  I.,  (1982).  Curiosity  and  the  interested  explorer   NSP1  Journal.  May,  19  22 

Fry,  J.  (1972).  Interactive  relationship  between  impiiHittvenoKK  and  student  control 
of  instruction.  Journal  of  Educational  PsycholoMy.  Ii&_5,  150  4(15 

Gay,  G  (19H(I)  Interaction  of  learner  control  and  prior  understanding  in  computer 
assisted  video  instruction.  Journal  of  EdlAf ilUuiUtl Vm llulniiX  V'/H.  ii  225  227 


14  p. 


ERLC 


llay,  11.,  Trumbell,  I) ,  and  Smith  J.  (1988).  Perceptions  of  control  anil  use  of 
control  options  in  computer-assisted  video  instruction.  Tech  Trend b.  V33»  3I-IW. 


Haunutln,  M  ( 1984)  Guidelines  for  using  locus  of  instructional  control  in  the 
design  of  computer-assisted  instruction.  Journal  of  Instructional  Development.  V7, 

3,  0-9. 

Ilaniiafln,  M.  and  Col  amnio  M.  (1987).  The  effects  of  variations  in  lesson  control 
and  practice  on  learning  from  interactive  video.  Educational  Communication  and 
Technology  .Journal,  V35,  4,  203-212. 

Henderson,  B.  B.  and  Gold,  R.  R.  (1983).  Intellectual  styles:  A  comparison  of  factor 
structures  in  gifted  and  average  children  and  adolescents.  Journal  of  Persqptility 
and  Social  Psychology,  V45,  3,  624-632. 

Johunsen,  K.  J.,  and  Tennyson,  R.  D.  (1983).  Effect  of  adaptive  advisement  on 
perception  in  learner-controlled,  computer-based  instruction  using  a  rule  learning 
task,  Educational  Communications  and  Technology  Journal.  V4t»  A>  226-236 

Langevin,  R.  (1971).  Is  curiosity  a  unitary  construct?  Canadian  Journal  of 
Psychology,  V25.  360  374. 

Laurillnrd,  I).  M.  M9H4).  Interactive  video  and  the  control  of  learning.  Educational 
Technology.  V26,  6,  7  15. 

Maw,  W  and  Maw,  E  (1961)  Establishing  criterion  groups  for  evaluating  measures 
of  curiosity.  Journal  nf  Experimental  Education,        £,  299  365. 

Maw,  W  and  Maw,  E.,  (1964)  An  Exploratory  Study  into  the  Measurement  of 
Curiosity  in  Elementary  School  Children,  Cooperative  Research  Project  No  801. 

Maw  W  and  Mnw  E  ,  ( 1965)  Personal  and  Social  Variables  Differentiating 
Children  With  High  and  Um  Curiosity.  Newark,  Delaware:  University  of  Delaware 

Maw,  W.  and  Maw,  E.  (1977)  Nature  and  assessment  of  human  curiosity  In  P 
McReynolds  (Ed.),  Advances  in  Psychological  Assessment.  V4,  San  Francisco; 
Jossey-Bass, 

Maw,  W  and  Maw,  E.  <  1965)  Differences  in  preference  for  investigatory  ucfivtlies 
by  school  children  who  differ  in  curiosity  level.  Psychology  in  the  Schools.  Y^,  263 

266 

Maw,  W  and  Maw  E  ,  (1966)  Children's  curiosity  and  parental  attitude*.  Journal 
of  Marriage  and  the  Eamily,        343  345. 

Merrill,  M  D.  (1975)   Learner  control  Heyotid  apt  it  tide- treatment  interactions 
/\ui|io  ViHi|Ml  Communications  Review,  V23.  217  226. 

Milheim,  W  D.  and  Azbell,  J.  W.  (1988).  How  past  research  on  learner  control  can 
Tiiil  in  the  design  of  interactive  video  materials.  Paper  presented  at  the  Association 
for  Educational  Communication  and  Technology,  New  Orleans,  Louisiana,  11  p. 


ERLC 


113 


Naylor,  F.  D.  (1981).  A  state-trait  curiosity  inventory.  Australian  Psychologist.  V10. 
%  172-183. 

Necka,  E.  (1989).  Stimulating  curiosity.  Qiftecj  Education  International,  VG,  25-27. 

Parker,  M.  and  Engel,  J.  (1984)  A  unified  motivation  and  learning  theory  model. 
Education,  V1Q3.  4,  353-300. 

Penney  R.  K.  and  McCauti,  B.  (1964).  The  Children's  Reactive  Curiosity  Scale. 
Psychological  Reports.  YJ5,  323-334. 

Pedhazur,  E.  J.  (1973).  Multiple  Regression  In  Behavioral  Research  ,  Explanation 
and  Prediction.  New  York:  Holt,  Rinehart,  and  Winston. 

Raskin,  D.  ( 1973).  Attention  and  arousal,  in  Prokasy  and  Raskin  (Eds.) 
Electrodermal  Activity  In  Psychological  Research.  New  York:  Academic  Press 

Reigeluth,  C.  M.  (1983).  Instructional  design:  What  is  it  and  why  is  it?  In  C  M 
Reigeluth  (Ed  J,  Instructional-Design  Theories  and  Models.  An  Overview  of  Their 
Current  Status  Hillsdale,  New  Jersey:  Lawrence  Erlbaum  Associates. 

Hilvorstein,  A.,  Pearson,  L,  Duntilck  If,  and  Ford,  T  (19HM.  Psychometric 
properties  of  two  measures  of  intrinsic  motivation.  Perceptual  and  Motor  Skills, 
V53,  055-658. 

Tennyson,  R.  D.  (1980).  Instructional  control  strategies  and  content  structure  as 
design  variables  in  concept  acquisition  using  computer-based  instruction.  Journal  of 
Educational  Psychology.  V72.  525  532. 

Tennyson,  R.  D.  (1981).  Use  of  adaptive  information  for  advisement  in  learning 
concepts  and  rules  using  computer-assisted  instruction.  American  Educational 
Research  Journal.  V18,  4,  425  438. 

Tennyson,  R.  D.  (1984).  Artificial  intelligence  methods  mi  computer  based 
instructional  design:  The  Minnesota  Adaptive  Instructional  System.  Journal  of 
Instructional  Development,  V3,  17  22. 

Tennyson,  R  I) ,  Chrfstensen,  I).  L  and  Park,  S  I    ( 19H4 )  The  Minnesota 
Adaptive  Instructional  System.  Journal  of  Compter  Based  Instruction,  VI 1.  2-13 

Vidler,  I)  C,  and  Rawan,  II.  R.  (1974).  Construct  validation  of  a  scale  of  academic 
curiosity.  Psychological  Reports,  V35,  263-266. 

Vidler,  II.  C,  and  Rawan,  H,  R.  (1975).  Further  validation  of  a  scale  of  academic 
curiosity  Psychological  Reports.  V37,  115-118. 

Wilson,  F.S.,  Stuckey,  T.,  and  Langevin,  U.  (1972).  Are  pupils  in  the  open  plan 
school  different?  The  Journal  of  Educational  Research.  V6(i,  3,  1 1 5- 1 1 8. 


114 


Title: 

Wide-Area  Network  Reeouroes  for  Teacher  Education 


Author. 
Ronald  Aust 


115 


ht  4 


Wide-Area  Network  Resources  for  Teacher  Education 


Ronald  Aust 
Associate  Professor,  University  nf  KiinKiiH 


Imagine  liNiilitt^  a  classroom  discussion 
and  being  able  to  immediately  displti y  any 
text,  instructional  softwure,  or  vidian 
resource  thnt  lit  the  direction  of  an  evolving 
debate.  Later  your  students  engage  In  an 
inquiry  activity  supported  by  information 
that  they  gather  from  immense  network 
data  bases  on  such  topics  as  population 
demographics,  employment  statistics  and 
longitudinal  national  weather  records. 
Finally,  you  consult  your  personalised 
electronic  newspaper  and  send  an  K-mnll 
uuestion  to  the  author  of  an  article  that 
describes  an  alternative  data  base  inquiry 
activity.  Such  scenarios  may  soon  be 
actualized  through  recent  advances  in 
computers  and  the  improving  climate  toward 
establishing  wide-area  national  networks. 

Oil  December  9,  1991,  the  president 
authorized  the  High  Performance 
Computing  (1  IPC)  Act.  For  1992,  legislature 
appropriated  nearly  $000  million  to  eight 
agencies  (l)AHPA,  NHK,  IM)K,  NAHA,  KPA, 
NlMT,  NOAA  and  Kdueatlnn)  in  support  of 
UPC  If  fully  appropriated,  federal  funding 
for  UPC  from  FV  92  to  FY  90  will  approach  :| 
billion  dollars.  A  central  functional  1IPC  is 
to  establish  the  necessary  policy  and 
resources  for  implementing  n  National 
Research  and  Kducatlon  Network  (NHKN) 
Among  other  goals  NHKN  will  seek  "..  to 
expand  the  number  of  rosea  reborn, 
educators,  and  students  with  training  in 
high-performance  computing  and  across  to 
high-performance  computing  resources;  and 
to  promote  the  further  development  of  an 
information  infrastructure  of  data  bases, 
services,  access  mechanisms  and  research 
facilities  ..."  (Public  Law  102  194;  section 
3,  II. 

NHKN  will  greatly  improve  (he  volume 
and  speed  with  which  the  rapidly  growing 
pools  of  electronic -based  information  are 
delivered.  Hy  1990,  NHKN  will  be  capable  of 
transmitting  at  least  a  billion  bits  (gigabits) 
per  second  lOnre,  1991),  thereby  opening  a 
new  gateway  to  a  vast  array  of 
elect  ran Ic  based  information  resources. 
What  level  of  access  do  touchers  and  teacher 
educators  need  to  benefit  from  this  electronic 


superhighway?  What  kinds  of  network 
resources  are  most  useful  for  educators? 
What  design  issues  are  crucial  for  the 
successful  implementation  of  networking  in 
schools? 

TJiiLL^iTJimtum 

This  paper  discusses  factors  to  consider 
when  developing  wide-area  network 
resources  for  educators.  Some  of  the 
recommendations  come  from  experience  and 
research  compiled  over  a  two  year  period  in 
association  with  the  Unified  Network  for 
Informatics  in  Teacher  Kducatinu  (UNITKl 
networking  project  that  was  initiated 
through  a  grant  horn  the  Apple  Corporation 
UNl'IK  is  a  prototype  tor  linking  educators 
who  an1  geographically  separated  but  share 
common  goals  tot  advancing  teacher 
education  HNITK  targets  tout 
rniiHtit neui'les  eoncci  lied  with  teacher 
ed neat  ion  student  InleriiH,  classroom 
teaclieis.  ad  ml  a  1st  I  a  lots  and  tea  diet 
cdihslojh  These  constituencies  exchange 
Mm  t  (trough  elect  i  on  Ic  mail  and 
c  ii  Mi  III  1 1  n  1 1  \  published  re  ho  n  lies  on 
instinct  tonal  soflwaie,  lesson  plans,  field 
trips,  mentor  profiles  and  reaeatch 
abstracts  The  UNITK  stations  ate  located 
in  the  school  s  lihiar\  or  staff  development 
urea  A  school  bused  Hainan  team, 
consisting  of  an  administrator,  a  teacher,  a 
librarian/computer  specialist  and  it 
available,  a  teacher  education  intern , 
monitors  use  of  the  stations  Student 
involvement  with  UN1TK  centers  on 
cross-school  discovery  activities  intended  to 
extend  data  collection  and  analysis  across 
content,  grade  levels  and  schools 

In  September  1989,  dedicated  UNITK 
stations  consisting  of  a  Macintosh  HK, 
modem  and  printer,  were  established  in  six 
secondary,  live  middle  and  five  elementary 
schools  representing  six  eastern  Kansas 
school  districts.  The  schools  connect  to  the 
hub  computer  at  the  University  of  Kansas 
for  updating  the  mail  and  resource  (lies 
Tht*  transmission  speed  is  slow  ( 1200  baud). 
However,  the  system  is  capable  of  emulating 
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the  immediacy  of  faster  trunKmiHHiun  h pood 8 
by  making  transfers  at  night  and  using  the 
local  hard  drives  for  storing  files,  The 
disadvantage  of  this  method  of  distribution 
iH  that  the  quantity  of  immediately  available 
resources  is  limited  to  the  size  of  the  local 
hard  drives.  AIho,  oh  a  prototype  that  haH 
seen  many  changes  to  accommodate  user 
preferences,  reliability  has  suffered  and  we 
ure  considering  "freezing"  the  system  in  a 
mine  dependable  configuration.  The 
following  recommendations  are  for  extending 
navigational  aids,  such  as  UNITE,  to  take 
advantages  of  the  growing  quantity 
resources  available  through  wide-area 
networks. 

Expectation  Mqnagomc n t 

Introduction  of  educational  innovation 
requires  a  delicate  balance  between 
enthusiasm  and  realistic  goals.  On  the  one 
hand,  administrative  support  is  needed  to 
launch  innovative  projects  hut  too  much 
enthusiasm  can  breed  unrealistic 
expectations.  Obtaining  this  balance  can  be 
especially  challenging,  given  the 
administrative  procedures  for  allocating 
projects  within  local  educational  agencies 
and  because  there  are  unique  reward 
systems  for  school  of  education  faculty, 
practicing  teachers  and  school 
administrators. 

During  the  initial  administrative 
orientation,  presenters  explained  that  the 
UNITE  resource  catalogs  begin  as  'empty' 
vessels  to  he  filled  over  time  by  the 
constituents.  Even  with  this  explanation, 
messages  conveyed  to  same  teachers 
suggested  that  a  complete  compendium  of 
resources  would  be  available  shortly.  When 
introducing  innovation,  developers  should 
consider  a  separate  session  for 
administrators  that  explicitly  deals  with  the 
importance  of  expectation  management. 
Network  representatives  may  also  wish  to 
stipulate  that  the  participating  schools  must 
demonstrate  an  intact  support  system  for 
com ni it er  maintenance  and  training  before 
establishing  wide-area  services. 

EHtiihljHhtniE  Lulu]  Liiikn  to  Wick-Anui 
Networks 

Each  ol  the  17  participating  K- 12  schools 
has  a  single  UNITE  station.  We  used  the 
approach  of  distributing  stations  across 
several  schools,  as  opposed  to  more  stations 
in  fewer  schools,  in  order  to  gain  feedback 


from  a  variety  of  grade  levels  in  urban, 
suburban  and  rural  settings.  As  the  project 
progressed,  an  increasing  number  of 
teachers  expressed  the  need  for  network 
connections  from  classrooms.  These 
teachers  explained  that  classroom  network 
access  would  allow  them  to  use  the  network 
more  frequently  and  offer  more 
opportunities  for  student  involvement. 

Many  advances  in  connecting  computers 
have  emerged  since  the  beginning  of  the 
UNITE  project  and  networking  cost  have 
decreased  significantly.  Currently,  the  best 
first  step  in  introducing  wide-area 
networking  may  be  to  establish  a  reliable 
Local  Area  Network  (LAN)  with  computers 
in  classrooms.  The  computers,  connections 
and  the  networking  software  used  in  forming 
a  LAN  vary  considerably.  Generally  the 
LAN  should  support  printer  sharing, 
electronic  mail,  access  to  a  central  server 
and  be  capable  of  exchanging  files  at  a  rate 
of  at  least  10  million  b*ts  per  second. 

There  are  several  advantages  in 
establishing  a  school-wide  LAN  before 
introducing  wide-area  networking  services. 
A  LAN  is  a  good  platform  for  familiarizing 
users  with  the  computers,  network  services 
and  the  training  provided  by  local 
networking  administrator.  If  the  local 
support  is  not  sufficient  to  maintain  a 
reliable  LAN,  it  will  not  support  extensive 
use  of  wide-area  services.  After  establishing 
a  reliable  LAN,  you  can  attach  a  network 
bridge"  to  allow  all  users  on  the  LAN  to 
access  to  wide-area  network  services. 

Wide-Area  Network  Services 

Many  of  the  regional  networks,  that 
center  on  universities,  government,  and 
large  businesses,  are  now  connected  to  form 
the  Internet  (the  precursor  to  the  proposed 
higher  speed  NREN).  Users  of  the  Internet 
can  exchange  mail,  make  use  of  the  higher 
processing  capabilities  of  super  computers 
and  access  resources  servers  across  the 
Internet,  The  number  id*  servers  on  the 
Internet  is  growing  exponentially.  These 
servers  allow  users  to  gain  access  to  any 
information  represented  in  a  digital  format 
including:  text  of  newspapers  and  online 
journals,  instructional  software,  terrain 
maps  based  on  satellite  imagery,  and  mote 
recently,  full  motion  digital  video  that  is 
compressed  in  digital  formats  such  as 
QuickTime1  M,  Home  of  the  fastest  growing 
servers  are  those  that  adopt  the  file  transfer 
protocol  developed  by  The  University  of 
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Minnesota's  "UopherQT  group,  another  Ilk1 
transfer  system  developed  nt  Dartmouth 
culled  "Fetch©"  and  the  text  miih rehinu 
system  for  accessing  information  from  the 
DowQues;  ,m  developed  hy  Thinking 
Machines  (Corporation  called  "WAIHtationlV 
Because  some  of  these  developers  openly 
encourage  other  to  establish  additional 
servers  that  use  their  protocol,  the  number 
of  Hervers  and  thus  the  overall  amount  of 
information  available  through  the  Internet 
is  likely  to  continue  to  grow  at  an 
exponential  rate. 

The  standard  for  routing  information  on 
the  Internet  is  the  Transmission  Control 
Protocol/Internet  Protocol  (TCPIP).  Each 
computer  using  TCPIP  must  have  a  unique 
IP  number  for  routing  information  requested 
from  servers  back  to  that  computer.  Thus, 
to  connect  a  LAN  to  the  Internet  the  school's 
network  administrator  will  need  to  acquire  a 
network  bridge,  a  high  speed  data  line  to  the 
Internet  f preferably  one  that  transmits  at 
leaBt  56  thousand  bits  peer  second 
f>6Kbp8),  and  a  series  of  unique  IP  numbers 
that  anticipate  growth  across  the  school  or 
district. 

Designing  Wide  Area  Networks 
Interfaces  for  Teacher  Education 

Even  with  (he  vast  amount  information 
available1  through  wide-area  networks,  it  will 
bo  important  that  teachers  and  teacher 
educators  I  in  mediately  recognize  I  he 
usefulness  of  available  resources.  Teachers 
need  a  navigational  aid  that  assists  them  in 
integrating  resources  into  their  curricula 
The  resource  navigation  component  of 
UNITE  (Figure  I)  serves  this  purpose1 
through  a  framework  for  accessing  and 
contributing  to  the  catalogs  of  dynamic 
resources.  Currently,  the  navigator  has  1(1 
content  hierarchies  (Language  Arts,  Math, 
Science...),  with  several  hierarchies  having 
over  100  sub-categories.  Each  content 
hierarchy  is  further  expanded  by 
cross-referencing  with  the  primary  support 
categories  of  administration,  courseware, 
research  and  teaching.  The  navigators  links 
to  Instructional  support  are  shown  as  leom: 
representing  the  resource  catalog:  for:  Fieid 
Trips,  Lesson  Plans,  Courseware 
Descriptions  and  Research  Abstracts 

Mesldes  assisting  teachers  in  integrating 
resource's,  we  assumed  Ihnt  the  resource 
navigation  system  must  he  easy  for  teachers 
to  use  Others  have  noted  the  need  (or 
developing  intuitive  Intcfaccs  in  public 


school  computing  networks  (Hull  et.  al., 
IfHIMl.  However,  a  design  that  one  group  of 
educators  finds  Intuitive1  may  be  difficult  for 
another  group.  For  this  reason,  we  adopted 
a  rapid  prototyping  (Tripp  &  Hichchneyer, 
1090,  Carrol,  I  HIM)  I  approach  to  development 
that  Involved  educators  from  several 
constituencies  in  evaluating  designs  (Aust  & 
Klayder,  If  ID  1 1. 

The  current  design  emerged  over  a  two 
year  period  and  included  five  version 
upgrades.  After  each  version  release,  users 
sent  recommendations  to  the  "Catalog 
Critics"  special  interest  group  on  the 
electronic  mail  system.  As  the  users  worked 
with  each  prototype  they  began  to  comment 
on  design  issues  and  common  preferences 
soon  emerged.  User  recommendations 
resulted  in  changes  to  the  icon  design  and 
placement,  consistency,  feature  mimes  and  a 
general  simplification  of  the  interface. 
Without  this  rapid  prototyping  approach  the 
design  of  the  resource  navigator  would  have 
been  considerably  different  and  likely  less 
useful  to  educators. 

The  recently  developed  navigation 
systems  by  the  (jnpherl>  (Figure  2)  and 
Fetch*"1  ( Figure  III  groups  also  provide 
insights  for  developing  effective  wide-area 
network  navigation  systems,  (topher  and 
Fetch  are  both  more  open  development 
environments  than  the  UNITE  resource 
navigator.  They  allow  the  server's  systems 
operator  to  establish  the  content  hierarchy 
those  best  suits  local  needs.  Clearly,  this 
strategy  has  been  successful  in  terms  of 
causing  a  proliferation  of  servers  across  the 
Internet.  However,  as  the  number  of  open 
content  hierarchy  servers  expands,  the 
number  of  wayfinding"  problems  that  cause 
users  to  become  lost  in  hyperspace  will  likely 
increase  as  well  (Kerr,  1980;  Kinzie  et.  al., 
1000). 

A  standard  for  organizing  leachei 
education  resources  on  wide-are  networks  is 
needed.  The  content  structure  of  this 
standard  should  accommodate  a  variety  of 
curricula  so  that  teachers  will  easily 
understand  how  the  resources  are  integrated 
with  their  exist  lug  teaching  strategies. 
Other  inlegories  to  consider  for  this 
standard  include:  media  format,  length, 
oppropiiate  grade  level,  prerequisite  skills, 
geographic  stipulators,  publisher,  and 
support  categories  such  as  classroom  use  or 
professional  development. 
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Community  PubllHhod  Dynamic 

The  UNITE  catalogs  of  instructional 
resources  are  dynamic  because  any  user  on 
the  system  can  expand  or  modify  them 
through  n  contribution  and  review  process. 
Koch  catalog  is  essentially  a  distinct 
data-base  consisting  of  tailored  fleldH  such 
as  subject  descriptor,  grade  level,  equipment 
requirements  and  publisher.  These  catalog 
fields  provide  necessary  Indexing 
information  for  conducting  multiple  key 
word  searches,  In  some  cases,  resource 
described  In  the  catalogs  can  be  previewed 
directly.  For  example,  after  reading  a 
description  in  the  courseware  catalog,  users 
can  immediately  preview  Instructional 
shareware  packages. 

The  development  of  the  dynamic 
resources  follows  recominendat ionH  for 
teacher  collaboration  and  empowerment 
(Goodlad,  1990;  Grady,  1988;  McDonald, 
1989;  MaerofT,  1988).  Instead  oi  Imposing 
top-down  "expertise,"  contribution  tools 
encourage  educators  from  all  constituencies 
to  participate  in  both  contributing  and 
reviewing  resources  Any  uhci*  can 
contribute  a  suggestion  by  first  selecting  the 
"Add  a  New  Resource "  or  the  "Modify  an 
Existing  Resource"  option.  Users  follow  a 
step-by-step  procedure  as  they  are  guided 
through  the  contribution  process.  When  a 
contribution  arrives  at  the  hub,  a  message  Is 
sent  to  all  reviewers  explaining  that  a 
resource  is  available  for  review.  Reviewers 
are  extended  throughout  the  system  and 
may  Include  representatives  from  all 
constituencies, 

Once  a  resource  is  requested  by  a 
reviewer,  il  Is  unavailable  to  other  reviewers 
until  released.  This  avoids  conflicting 
reviews  During  the  review  process  the 
reviewer  and  author  often  communicate 
through  electronic  mail  to  exchange 
suggestions  or  resolve  concerns  Once 
approved,  a  new  or  modified  resource 
becomes  available  across  the  network  foi  use 
and/or  further  modification.  Names  of  the 
original  author  and  reviewer  uh  well  as  the 
names  nl  any  other  contributors  or  reviewers 
are  listed  with  each  resource  entry 

An  evaluation  of  instance  usage  patterns 
and  ojisei  villoma  of  I  IN  IT  K  users  revealed 
several  inn  iff  hi  a  concerning  t  he  dse  ot 
comnulei  based  resnuires  by  educators 
Mi mI,  many  users  expect  vast  amounts  of 
information  under  each  resource  category 
and  If  their  favorite  category  is  low  In 


resources  they  lose  interest  quickly.  Also, 
different  constituencies  prefer  different 
resource  categories.  For  example,  teacher 
education  majors  prefer  catalogs  of  lesson 
plans  whereas  practicing  teachers  prefer 
resources  of  a  timely  nature  or  regional 
specialty,  Some  of  the  most  often  used  and 
best  received  resources  are  those  developed 
for  local  uho,  such  as  data  bases  of  field  trips. 
Searching  mechanism  for  large  scale 
database  should  include  indexes  (e.g.,  by 
county)  for  tailoring  searches  to  regional 
interests. 

Regarding  community  publishing,  users 
who  publish  on  networks  become  vested  and 
are  more  likely  to  continue  network  use.  In 
a  vote  during  one  UNITE  project,  teachers 
unanimously  preferred  a  peer  review,  as 
opposed  to  an  administrator  controlled, 
process  for  monitoring  publication  on  the 
system.  We  also  found  that  the  teacher 
education  constituency  most  likely  to  freely 
contribute  resources  are  teacher  education 
students,  MoHt  teachers,  on  the  other  hand, 
will  expect  some  form  of  compensation  for 
contributing  resource  entries.  This 
compensation  may  be  in  the  form  of 
monetary  reward,  release  time,  course  credit 
or  addit  ional  access  to  networking  resources. 

Hocial  and  Administrative  Considerations 

Many  of  the  initial  social  and 
administrative  barriers  to  electronic 
networks  may  be  overcome  by  providing 
direct  and  immediate  access  for  all  teachers. 
After  conducting  a  naturalistic  study  (Ouha, 
1981)  on  the  implementation  of  UNITK, 
Hlchelmeyer  I  1992)  concluded  that  the 
Integration  of  technology  In  schools  Is  often 
constrained  by  it  Teacher  Needs  Hierarchy, 
This  hierarchy's  contingency  schedule  Is 
similar  to  Maalnw's  Human  Needs  Hierarchy 
where  level  one  must  he  reached  before  level 
two  is  reached  and  so  on.  Him  proposed  the 
following  steps  hi  the  Teacher  Needs 
Hierarchy  (I)  Time  and  Accessibility,  lU) 
Dependability,  till  Vesting,  (41  Control,  (5) 
Integration. 

As  t'lilmii  ( It)H(J)  documented,  the  Initial 
enthusiasm  foi  technological  innovation 
often  subsides  rapidly  If  measures  are  not 
taken  to  manage  expectations  and  renew 
Interests.  Much  of  the  local  enthusiasm  is 
often  associated  with  Individual  advocates 
whose  status  or  popularity  influences 
success.  When  these  individuals  leave  a  site 
the  enthusiasm  for  "their"  innovation  often 
leaves  with  them.   This  Is  especially  true 
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when  Incoming  technology  mentors  seek  to 
make  their  mark  by  adopting  and  advocacy 
lhat  differs  from  the  predecessor.  Advocates 
should  attempt  to  extend  enthusiasm  ho  that 
all  participants  have  ownership  in  the 
innovation  -  an  approach  that  may  require 
significant  restructuring  of  school  s  reward 
system. 
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Figure  1 

The  UNITE  Resource  Navigator 


Content  categories  appear  vertically  along  the  loft  margin  of  the  screen.  The  lour 
support  categories,  Administration,  Courseware,  Teaching  and  Research  appear 
horizontally  along  the  hottotn  of  the  resource  navigator.  In  this  view  the  UBer  began  by 
selecting  tin1  support  category  for  Courseware  followed  by  the  main  content  category 
Natural  Science  and  then  the  sub-categories  of  Life  Science,  Ecology,  Ecosystems  and 
Habitats. 

In  this  view  the  user  is  about  to  selected  the  software  Kansas  Keptilvs  Menu  listed 
under  the  Courseware  support  for  the  selected  content  hierarchy.  The  dark  icon  in  front 
of  tin*  listing  indicates  that  the  user  will  be  able  to  directly  preview  this  software  on  the 
UNITE  system. 
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Figure  2 

The  Gopher®  Navigational  Interface 


#  File  Edit   Gopher  Setup  Recent 


Bookmarks 


Home  Gopher  Seruer 


News 


National  Weather  Seruice  forecasts 


Kansas 


Topeko 


topeka  and  vicinity  forecast 
national  weather  service  topeka  ks 
1050  am  est  sun  jan  10  1993 

this  afternoon,  .cloudy,  cold,  occasional  flurries,  high  20  to  25. 
northeast  wind  5  to  10  mph. 

tonight     .cloudy    low  in  the  teens,   light  northeast  mind. 

monday  ..cloudy,  a  50  percent  chance  of  snow  or  sleet,  high  in  the 
upper  20s.  southeast  wind  10  to  15  mph. 

extended  forecast... 

tuesday.    a  chance  for  snow. .  except  freezing  rain. .  sleet  and  snow 
In  the  southeast,  cold  with  lows  5  to  10  west. .10  to  15  north, 
near  20  southeast  corner,  highs  in  the  teens  to  lower  20s... except 
near  30  southeast. 

Wednesday.,  a  chance  for  snow  early  In  the  extreme  east,  colder 
with  lows  near  zero  In  the  northwest  to  the  teens  In  the  southeast, 
highs  In  the  teens  to  lower  20s. 


The  (iopheKL)  software  and  protocol  were  designed  by  the  Gopher  team  at  the 
University  of  Minnesota.  Gopher  in  a  fast  and  simple  way  fur  searching  and  distributing 
files  that  has  gained  considerable  popularity  on  the  Internet.  Each  server  administrator 
determines  the  structure  of  the  content  hierarchy  on  their  server,  (lopber  databases 
usually  use  full-text  indexes  meaning  any  word  in  the  database  is  a  key  word. 

In  the  above  depiction  the  overlapping  windows  show  the  path  the  user  took  to 
obtain  a  regional  weather  report  from  the  national  weather  service  on  the  home  (lopber 
server.  The  National  Weather  Service  data  base  is  updated  frequently. 


Figure  H 

The  Fetch©  Navigational  Interface 


i  Hie  Edit  Remote  Directories  Customize  Windows 


Fetch:  dartuaH.dGrtmouth.9du 

wmammmmt 

t?o  t  n  K     Copyr1gh\  1'  1 992 

i  V?  t^H     Trustees  of  Dartmouth  College 

Close  Connection  Kill] 

pub  ▼] 


CD  6300+ 

Q  dnd.tar.Z 

CD  dropoff 

PD  gnuplot 

Q  ICMA-Libr... 

CD  mac 

CD  PTSD 
0  rdate.c 
D  README 
Q  rn.4.4.tar.Z 


-  Dec  19  11  :23  [O; 

-  Feb  ?  1992 
58K  Apr  9  1992 

-  Jan  10  23:47 

-  Dec  2  05  :29 

-  Dec  1 4  20 :06 

-  Dec  28  1 7 :09 

-  Jul  10  1992 
6K  Mar  9  1992 


2K  May  12  1992 


Put  File. 


(  Get  File, 


<§)  Automatic 


Status 
Connected. 

Eik 


Transfer 


The  Fetch©  system  was  developed  at  Dartmouth  College.  Fetch  uses  the  File 
Transfer  Protocol  (FTP)  to  transfer  files  on  the  Internet.  The  interface  is  similar  to  the 
one  used  by  application  programs  to  locate  Hies  on  a  hard  drive. 
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Schools  have  always  icmved  wimc  measure  of  community  support,  such  as  parental  assistant e. 
voluutt'ei  HIomk,  donations  ol  money  and  equipment  and  provision  ol  needed  expertise  Hui  fueled  hv 
ivporiN  such  ah  a  MiUiilU  lU  liiiik,  calls  for  school  reform  by  almost  every  major  political  candidate  and 
i  imeless  media  accounts  of  apparent  educational  failure,  there  is  a  growing  public  perception  thai 
A  me  i  lean  schools  are  In  a  trusts  professional  educators  cant  Tlx  by  themselves  One  positive  lesult  of  this 
perception  is  mote  oigani/rd  and  substantial  involvement  in  schools  by  influential  community  institutions, 
which  band  together  in  "educational  partnerships"  to  provide  resources  and  sei  vices 

Such  educational  partnerships  aiv  increasing  dramatically.  There  nie  thousands  of  them,  big  and 
small,  and  nunc  all  the  nine;  the  National  Alliance  id  Uusmess  (NAH)  estimated  140,1)00  in  I'JKY  As 
might  Ik?  expected  wiih  a  ^towing  phenomenon,  these  parnVrships  aie  as  diverse  in  their  purposes  as  they 
are  in  their  membership.  Small  collaborations,  such  as  those  between  a  single  local  school  and  a  neighbor 
hood  business,  are  usually  focused  on  providing  a  particular  service  lor  a  specific  school  group,  such  as 
business  mentors  for  science  students  or  tutors  for  at-risk  kids,  or  they  may  provide  in-kind  assistance 
such  us  free  printing  ol  school  materials.  They  may  also  donate  items  to  the  school.  Larger  partnerships, 
involving  bigger  businesses  or  corporations,  and/or  universities  and  groups  of  schools,  usually  have  more 
ambitious  and  far-tanging  goals,  including  effecting  comprehensive  school  reform. 

lor  many  reasons,  the  goals  of  these  larger  educational  partnerships  often  include  the  purchase  of 
educational  technology  and  promotion  of  its  integration  into  the  daily  life  of  schools.  Kducaiional  tech- 
nology is  new  and  exciting,  has  a  high  public  profile  and  is  often  cited  in  popular  and  professional  liter- 
ature as  a  catalyst  tor  major  changes  in  leaching  and  learning.  At  the  same  lime,  however,  technology  can 
be  expensive  and  adopting  it  can  be  complex,  so  that  even  if  it  is  thought  desirable,  the  purchase  and 
effective  use  of  educational  technology  may  tax  the  means  and  experience  of  individual  schools  or  school 
districts.  10 therefore  likely  that  professionals  in  businesses,  universities  and  schools  interested  in  ciluca 
tionai  technology  will  find  themselves  reaching  out  to  and  working  with  other  community  groups  to  make 
possible  together  what  might  not  be  possible  alone. 
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l-cir  i hose  contemplating  such  collaUjiatmmi  utul  those  currently  involved  in  one,  it  would  be  helpful 
10  know  the  experiences  ol  others  A  dcsuiption  ol  what  has  worked  in  educational  technology  partner- 
ship!* from  the  point  ol  view  ol  ihose  who  have  been  involved  with  one  should  therefore  be  very  useful, 

Vil  unfortunately,  while  ipuir  a  hit  has  been  written  about  educational  partnerships  in  general,  not 
very  much  ol  it  has  been  n|hviIu  to  technology.  This  Is  hue  whether  you  examine  the  business,  educa- 
tional or  technology  management  llteiatuiv 

I  he  older  business  literatim1  lends  to  locus  on  the  peiccplion  that  schools  are  doing  an  inadequate  job 
piepanng  Ihe  tuiure  worklorce  as  a  lationale  lor  business  initiating  involvement  in  education  through  part- 
nerships. More  recent  analyses  olten  tetlect  a  clear  eyed  reassessment  of  the  efficacy  of  partnership 
aimugcmcniK,  hequently  citing  corpoiatc  frustration  with  vlow  or  superficial  change  and  a  luck  of 
accountability'1  for  resources  used  l.see  Cluon-  Kennedy,  Udelstein,  Mann,  Rist)  Both  the  early  and  the 
iccent  analyses  sometimes  reter  in  passing  to  a  lack  of  technical  sophistication  among  today  s  students  but 
they  generally  don't  single  out  technology  or  technology  partnerships  for  special  notice. 

Educational  analysts  largely  divide  into  two  camps,  those  who  consider  partnerships  as  one  more 
form  of  school  change  initiative  and  those  who  consider  them  as  policy-making  bodies.  Like  business 
observers,  school  change  writers  do  not  distinguish  technology  partnerships  from  other  collaborations. 
Instead  these  writers  point  to  the  difficulty  of  effecting  comprehensive  and  lasting  change  regardless  of  the 
innovation.  (See  Fullan.  Liebennan  and  Miller,  Sarason,  Sengc,  Sirotnik).  An  exception  here  is  the  work 
of  Larry  Cuban,  who  focuses  on  the  lailure  of  teachers  to  adopt  educational  technology  in  significant  ways 
(see  Cuban).  However,  Cuban  docs  not  speak  to  partnerships. 

liducational  policy  analyses  consistently  emphasize  that  no  matter  who  does  it,  educational  policy- 
making is  largely  political,  as  the  pioeess  ot  determining  "who  gels  what,  when  and  how"  is  essentially  a 
process  of  exercising  power  and  influence  no  matter  what  the  reasons  or  who  the  involved  parties,  (see 
Campbell  and  Maitzom,  liuston,  Lasswell,  Pawky).  Thin  literature  is  just  beginning  to  speak  to  partner- 
ships as  policy-making  bodies  and  so  does  nol  separately  consider  technology  partnerships. 

Last,  the  technology  management  literature  has  a  suhnianiial  group  of  analyses  concerned  with  both 
technology  and  partnerships.  But  these  studies  ate  largely  focused  on  examining  the  efficacy  of 
university/corporate  research  and  development  collaborations  and  their  success  in  developing  and 
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promoting  commercially-viable  and  competitive  technologies  (see  hissm.  MeHiicity  unil  t I  NrilK  mul 
Phillips).  The  findings  of  these  studies  might  be  relative  to  educational  partnerships,  at  least  where 
universities  arc  involved  with  schools,  but  that  connection  has  not  yet  t>cen  dhectly  diawn 

To  Kumnian  e,  while  it  would  be  beneficial  for  piofesslonuls  in  business,  umvci  sides  and  schools 
who  might  work  together  to  know  what  factors  help  lead  to  success  in  educational  partnerships  involving 
technology,  not  much  research  has  yet  been  done  which  is  specifically  helplul. 
Ueseareh  Questions 

i  he  major  research  question  for  this  then  study  is: 

What  are  lite  common  features  of  successful 
technology  partnerships? 

Related  research  questions  are: 

How  is  success  defined  mid  measured? 
Which  features  are  identified  as  successful  most  often? 
Which  features,  are  identified  as  problematic  most  often? 
Does  the  nature  of  the  group  involved  (l,e,,  business,  school, 

university)  make  a  difference  in  which  features  are 

identified  as  successful  or  problematic? 
What  suggestions  for  success  are  made  by  partnership  participants? 

The  wording  of  these  questions  indicates  an  emphasis  on  identifying  lactors  which  contribute  to  the 
success  of  technology  partnerships  from  the  point  of  view  of  those  currently  involved  in  one.  The  intent 
of  these  questions  is  therefore  descriptive  and  pragmatic  rather  than  conceptual  or  theoretic,  It  is  hoped 
answers  to  these  questions  will  be  of  practical  help  to  professional  people  contemplating  or  involved  in 
partnerships.  It  is  only  just  now  that  many  partnerships  are  of  sufficient  duration  that  then  participants  can 
assess  their  progress  and  draw  conclusions  to  share  with  others,  so  answers  to  these  questions  should 
prove  timely  and  helpful. 
Method 

Preliminary  analysis  was  done  on  a  pool  of  professional  literature,  journal  atttcles,  grant  icpoiis  and 
other  assessment  documents  of  educational  partnerships.  The  pool  resulted  from  a  comprehensive  review 
of  education,  business  and  technology  management  literature,  and  published  Department  ol  l  iducation 
(DOE)  grant  reports.  (SeeDanzberger  for  example)  These  articles,  grants  and  repot  is  weie  examined 
closely  to  see  what  features  of  partnerships  they  identified  as  helpful  or  problematic  and  what  sti^estions 
they  made  for  improvement. 
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The  results  ot  this  preliminary  analysis  pointed  to  the  following  elements  as  contributing  in  success 
shared  vision;  clearly-defined  goals;  an  institutionalized  decision-making  structure;  local  decision  making; 
continuity  among  partnership  personnel;  allowing  sufficient  time  for  change  10  occui ;  and  piovision  ol 
professional  development  time  and  uaining  to  teachers.  This  analysis  also  suggested  (hat  as  outside  policy 
making  bodies  alien  charged  wilh  allocating  much  needed  resources  and  composed  of  people  from  dis 
parate  institutions  with  dillerem  standards,  operating  procedures  and  goals,  ("culiuie"),  purmeislupb  hold 
the  |Mitential  (or  much  innllici,  (See  Sitotmk  rur example) 

However,  these  pielimiuaiy  results  hud  some  limitations.  Hirst,  materials  specilic  to  technology 
paruiershlps  weie  limited  Second,  the  reports  and  articles  which  were  available  were  universally  positive 
in  lone  and  lai  more  likely  in  identify  successful  features  than  problematic  ones,  probably  reflecting  a 
natural  tendency  to  play  up  successes  when  reporting  publicly  on  u  project.  Yet  it  could  be  as  helpful  for 
others  to  know  what  didn't  work  as  what  did.  Third,  the  documents  examined  often  didn't  distinguish  the 
disparate  views  of  particular  participants.  Instead  they  were  consensual,  making  it  impossible  to  determine 
if,  for  example,  the  business  partner  disagreed  with  the  school  partner  on  what  was  successful  and  why. 
Last,  because  there  were  not  very  many  summary  articles  and  reports  readily  available,  few  were  the 
results  of  systematic,  formal  research  into  partnerships,  and  identification  of  features  which  lead  to  success 
was  not  a  major  emphasis  of  much  of  what  was  written,  the  preliminary  analysis  was  based  on  a  relatively 
small  amount  of  data.  For  these  reasons  a  second  method  was  used  to  elicit  more  complete  information. 

Additional  data  was  collected  nationally  by  a  45-minute,  open-ended  telcjphone  interview  with  23 
representatives  of  15  educational  partnerships.  Each  had  at  least  two  years'  experience  with  a  partnership, 
collaboration  or  consortium  of  some  complexity,  involving  at  least  two  panics  and  administration  of  a 
large  federal  grant  specifically  focused  on  educational  technology.  (DOE  FIRST  I  Kind  for  the  Improve- 
ment and  Reform  of  Schools  and  Teachingl  grants,  with  an  average  award  of  $1 17,214  per  year  for  two 
or  three  years.)  For  most  of  the  respondents  1992  marks  the  final  year  of  a  three  year  grant  which  began  in 
1989.  For  some,  the  grant  ended  last  year  or  this  past  summer,  so  each  has  completed  or  is  completing 
their  project,  Many  had  written  or  contributed  to  one  or  more  assessment  reports  as  part  of  their  grant 
reporting  requirements,  and  so  had  the  experience  judging  the  success  of  their  projects. 
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All  23  respondents  were  somehow  involved  wilh  managing  the  federal  gram  and/or  the  partnership 
related  to  it,  being  the  ones  empowered  by  their  institutions  to  make  decisions  and  allocate  resources  tor 
shared  partnership  activities,  hive  people  were  school  district  representatives  of  some  kind,  live  repre- 
sented Corporations,  eleven  were  from  universities  and  two  were  I  mm  state  01  regional  education  depart- 
ments or  centers. 

Results  n|  the  pivllmmaiy  analysis  were  used  along  with  the  study  research  questions  lo  guide  design 
ol  the  interview  questionnaire,  which  solicited  comments  on  how  each  partnership  organized  collaborative 
work,  how  the  representative  felt  about  being  pan  of  a  partnership,  how  the  partnership  evaluated  success 
and  what  features  aided  or  impeded  success.  Respondents  were  also  asked  to  stale  their  advice  to  |>eople 
beginning  partnerships,  (see  Appendix  lor  interview  protocol) 

Statements  in  response  to  the  questionnaire  were  examined  lor  similarities  and  then  grouped  into 
categories  to  make  dominant  patterns  apparent.  Iliese  patterns  are  discussed  in  the  results  section  below, 
presented  along  with  some  individual  statements  which  were  not  widely  replicated  but  appear  nonetheless 
interesting  or  insightful. 
Results 

There  is  little  question  that  professionals  from  all  three  groups,  universities,  businesses  and  schools 
or  school  organizations,  generally  enjoyed  being  pan  of  a  partnership.  Many  respondents  described  their 
participation  as  "stimulating",  "challenging "  and  even  "fun",  presenting  a  definite  "change"  from  then 
usual  activities  which  afforded  them  "opportunities"  for  both  personal  and  professional  growth  Most  also 
lelt  greatly  rewarded  by  their  contact  with  a  new  set  of  "colleagues",  whom  they  often  described  as 
"friends"  as  well  as  "innovative",  "creative"  and  "exciting"  peers. 

About  a  third  of  the  respondents  saw  the  partnership  as  a  means  to  expand  their  range  of  personal 
"connections"  and  "professional  opportunities".  A  third  also  appreciated  their  chance  to  be  involved  in  a 
"real"  project  and  watch  the  educational  process  unfold  from  an  "insider's  view"  rather  than  from  a  dis- 
tance as  an  outside  professional  observer  or  lay  citizen.  And  almost  half  found  pride  and  gratification  that 
they  had  been  part  of  something  which  "made  a  difference",  elicited  "good  feedback"  or  had  an  "impact" 
on  education.  (Categories  are  not  mutually  exclusive.  Respondents  could  give  two  or  more  answers  to  a 
single  question  and  so  be  counted  in  more  than  one  category). 

6 

ERIC 


Respondents  also  liked  being  part  of  a  partnership  for  the  unique  benefits  they  felt  collaboration 
brought,  including  much-needed  "money  and  other  resources",  the  "synergy"  afforded  by  sharing 
"information",  "ideas"  and  "purposes"  and  the  "new  perspectives"  brought  to  educational  problems  and 
circumstances  when  multiple  people  from  disparate  backgrounds  interact. 

In  addition,  over  half  the  respondents  signaled  their  intent  to  continue  the  partnership  in  some  way 
after  the  initial  three-year  grant  period  ends  and  federal  monies  disappear.  Some  of  these  were  writing 
other  grants  for  support,  while  others  planned  to  absorb  ongoing  costs  among  involved  Institutions,  Of  the 
1 1  respondents  who  said  "no"  or  "mayl)e"  to  continuing  their  pai  met  ships,  none  said  they  would  never 
participate  in  another  partnership  and  most  were  actively  soliciting  new  partners  for  new  purposes.  The 
general  consensus  on  participating  in  partnerships  appeals  to  be,  as  two  tespondents  put  it,  "(in  for  it!" 
Measuring  Success 

However*  because  participants  genet  ally  en  joyed  being  part  of  a  partnership  does  not  mean  their 
partnerships  were  successful,  so  how  did  partners  define  and  measure  success?  This  question  is 
particularly  important  in  light  of  the  preliminary  analysis,  which  suggested  thatfeffferences  in  institutional 
cultures,  which  affect  what  kinds  of  results  are  valued,  typical  reward  structures  and  standard  operating 
procedures,  and  differing  institutional  agendas  for  participating  in  partnerships  can  cause  serious  dis- 
agreements about  what  constitutes  success  and  how  to  recognize  when  it  is  reached. 

In  this  group,  only  two  respondents  were  not  sure  if  their  project  had  been  successful  overall,  one 
citing  that  she  was  too  "close  to  it  to  tell  objectively",  and  the  other  that  he  regretted  "they  could  only  go  so 
far  since  it  was  important  not  to  go  too  last."  The  21  others  described  their  projects  variously  in  laudatory 
terms  such  as  a  model  others  can  use",  "1 10%  effective"  or  "it  reinforced  our  initial  beliefs." 

15  ot  the  respondents  stated  that  then  projects  used  specific  concrete  products  to  gauge  success, 
including  improvement  in  student  test  scores,  production  of  curriculum  units  and  lesson  plans,  develop- 
ment of  multi-media  protoypes  or  educational  software,  and  measurements  of  technology  usage,  such  as 
hours  spent  in  computer  labs,  or  cost  of  on-line  connect  time,  The  others  used  either  accomplishment  of 
grant  goals,  such  as  attainment  of  "objectives"  or  "outcomes"  or  adherence  to  "benchmarks"  and  "lime- 
lines",  or  qualitative  measures  of  student  and/or  teacher  attitudes  and  behavior,  such  as  "comments", 
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"feedback"  and  "evaluations  .  as  measures  of  accomplishment.  Many  had  a  mix  of  ways  to  gauge  their 
success,  including  formal  evaluation  by  paid,  outside  evaluators,  often  university  stall. 

Bvcn  though  the  majority  of  respondents  felt  their  project  was  successful  overall,  they  nonetheless 
expressed  reservations  about  how  evaluation  was  done,  6  persons  saying  they  either  didn't  agree  with  the 
methods  used  to  gauge  the  success  of  their  projects  or  weren't  sure  of  their  effectiveness,  and  many  of  the 
17  others  agreeing  overall  with  the  methods  used  but  with  some  specific  concerns.  These  concerns  were 
divided  about  equally  between  those  who  wished  more  traditionally  rigorous,  quantitative  assessments  of 
progress  and  those  who  thought  qualitative  measures  more  appropriate.  Comments  indicating  a  desire  for 
more  rigor  included  "need  more  evaluation  at  the  school  level-  it  was  hard  to  document  what  the  teachers 
were  doing",  "there  was  no  control  group",  "I'd  prefer  more  concrete  benchmarks",  and  "there  were  little 
pre-  and  post-test  differences  ",  while  those  arguing  for  more  qualitative  measures  stated  "it's  hard  to 
measure  long-term  goals",  "I'd  prefer  to  judge  investment  in  people,  not  scores",  "traditional  evaluation 
took  too  much  time  for  the  results  gained",  "I'd  add  qualitative  measures  of  attitudes  or  the  energizing 
effect  of  collaboration",  and  "usual  methods  are  merely  'add-ons'  at  end  of  the  project,  hut  no  one  will 
give  you  grant  money  for  formative  evaluation." 

The  concerns  expressed  about  the  effectiveness  of  evaluation  in  this  group  suggest  that  measuring 
success  is  one  of  the  more  problematic  aspects  of  a  partnership.  While  on  the  one  hand  partners  are  con- 
vinced they  have  been  largely  successful,  on  the  other  hand  they  are  not  satisfied  with  how  that  is  proved. 
ITieir  measures  of  success  often  appear  to  be  at  odds  with  their  perceptions  of  success.  This  may  be  be- 
cause participants  feel  torn  between  their  generally  positive  inside  experience  of  collaboration  and  the  need 
to  justify  its  fruits  to  influential  outsiders,  such  as  grant  officers,  tenure  committees  or  school  district  ad- 
ministrators, in  terms  they  can  accept.  Or  it  could  be  that  differing  cultural  expectations  of  partnership 
institutions  do  mean  differing  perceptions  of  measuring  success.  Or  it  could  be  something  else  altogether. 
This  study  does  not  sulficiently  address  why  these  differences  occur,  but  only  points  out  that  they  do,  just 
as  the  preliminary  analysis  suggested  they  might,  which  is  something  those  contemplating  a  partnership 
should  consider  for  frank  discussion  and  negotiation  among  partners  when  deciding  how  to  evaluate  their 
projects. 
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Problematic  Features 

What  features  of  the  partnership  were  identified  as  problematic  most  often?  Answers 
10  this  research  question  came  iiom  three  sources,  the  responses  to  questions  about  what  partnership 
features  impeded  success,  what  purl  nets  didn't  like  uhout  being  in  a  partnership  and  what  things  purtners 
disagreed  about. 

Failures  of  plaitnittK  and  implementation 

Failures  of  planning  were  mentioned  most  often  as  impeding  success,  including  picking  project 
"school  districts  too  far  apart  geographically",  having  a  "too  fast  production  schedule"  or  a  "contract  too 
binding",  "contradictions  in  project  goals",  "too  much  money  spent  on  hardware  with  none  for  training" 
and  "not  designing  technology  centrally  enough  into  the  project". 

It  is  relatively  easy  to  recognize  failures  of  planning  after  the  fact  when  what  hasn't  worked  becomes 
apparent.  It  is  much  harder  to  anticipate  problems  as  you  are  beginning  a  project,  especially  when  techno- 
logy partnerships  are  a  relatively  new  phenomenon,  local  conditions  vary  dramatically  and  there  are  few 
established  guideposts  to  follow.  As  one  respondent  said,  "This  is  an  experiment,  and  we  are  learning  as 
we  go  along."  ITie  simplest  conclusion  to  be  drawn  from  these  experiences  then,  is  to  devote  as  much 
time  as  possible  to  planning,  trying  to  anticipate  problems,  for  "the  more  you  plan,  the  easier  it  is  to 
implement".  But  recognize  there  will  be  many  problems  which  cannot  be  anticipated  or  controlled, 

Planning  problems  were  followed  in  importance  by  implementation  pioblems  such  as  excessive 
"changes  in  personnel",  and  introducing  the  project  on  too  grand  a  scale  with  "the  big  pitch'  which  scared 
school  participants". 

Some  of  these  barriers  to  success  can't  be  anticipated  or  well  controlled  either,  such  as  excessive 
turnover  in  personnel,  which  was  also  mentioned  prominently  among  things  respondents  didn't  like  about 
being  part  of  a  partnership.  When  people  change,  it  takes  time  to  "know  the  new  ones",  or  "bring  them  up 
to  speed",  and  when  key  people  go,  they  sometimes  "take  the  commitment  of  their  institutions  and  their 
resources  with  them".  Yet  while  participants  can  discourage  each  other  from  leaving  partnerships,  they 
can  t  prevent  each  other  from  going,  particularly  since  participation  in  the  partnership  is  seldom  anyone's 
main  job,  and  most  need  to  meet  the  demands  of  their  primary  job  first.  1  ^rsonal  and  institutional  agendas 
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change  through  time,  and  preservation  of  the  original  membership  in  the  partnership  is  rarely  more  import 

ant  to  participants  than  these  other  agendas. 

Stress 

Irt  discussing  what  they  didn't  like,  many  of  the  respondents  also  decried  the  stress  associated  with 
belonging  to  a  partnership,  stress  mostly  linked  to  the  amount  of  time  and  emotional  energy  that  collab- 
oration requires.  Many  partnership  participants  felt  their  institutions  did  not  provide  them  with  sufficient 
personal  support  for  their  positions,  mentioning  that  they  needed  more  "clerical  help",  more  "staff"  or 
"assistants",  and  most  importantly,  "time",  particularly  release  time  from  their  regular  institutional  duties. 
In  a  few  cases,  participation  in  a  partnership  was  even  added  on  top  of  someone's  other  job  duties  with  no 
extra  support  at  all,  "I  inherited  it  when  someone  else  left." 

One  corporate  officer  and  one  university  professor  both  felt  they  deserved  a  "promotion"  for  the 
work  they'd  put  in  but  clearly  didn't  expect  that  their  institutions  would  reward  them  that  way  for  their 
participation.  It  appears  partnerships  are  still  viewed  as  peripheral  commitments  by  many  organizations, 
and  their  success  relies  quite  a  bit  on  the  willingness  of  involved  parties  to  go  beyond  what's  provided  out 
of  personal  commitment. 
Logistical  problems 

On  a  more  basic  level,  respondents  also  frequently  mentioned  persistent  logistical  problems  which 
arose  from  the  complexity  of  group  managing  a  large  project.  These  ranged  from  major  decisions  such  as 
deciding  who  would  "manage  the  money"  or  be  "responsible  to  outside  auditors",  to  minor  but  consist 
ently  annoying  hassles  over  details  such  as  "conflicting  school  and  corporate  calendars",  "diflerent 
business  office  procedures  and  techniques",  and  "free  parking  at  the  university".  One  university  grant 
coordinator  pointed  out  that  her  university  was  not  used  to  producing  anything  but  written  materials,  and 
so  its  procedures  couldn't  easily  support  her  work  with  outside  multi-media  producers  who  required  large 
amounts  of  cash  and  rapid  production  and  payment  schedules  to  do  their  work. 

Respondents  found  that  resolving  these  logistical  challenges  took  a  surprising  amount  of  time,  yet  if 
left  untended,  could  erupt  into  bad  feelings.  One  school  district  technology  coordinator  laughingly  referred 
to  trying  to  park  his  car  at  the  university  to  attend  partnership  meetings  without  getting  a  ticket  he  had  to 
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personally  pay  as  a  "constant  thorn  in  his  side"  which,  although  minor,  was  a  reminder  to  him  that  he  was 
"not  on  my  turf  ". 

These  differences  in  routine  institutional  procedures  were  surface  manifestations  of  more  profound 
hiKliitilional  differences  which  often  caused  considerable  conflict,  Iliese  differences  surfaced  in  respondent 
answers  about  disagreement  among  partners.  Only  four  of  the  23  respondents  reported  no  conflict  of  note, 
one  explaining  that  disagreements  varied  by  the  people  involved  with  "no  major  consistent"  pattern  and 
another  adding  that  their  small  working  group  made  "agreement  more  possible  than  it  would  be  among  a 
large  number  of  people".  All  of  the  others  reported  some  disagreements. 

In  fact,  the  representatives  of  one  project  contacted  declined  to  participate  altogether  because  their 
collaborative  work  had  proved  so  difficult  the  first  several  years  that  in  the  interest  of  remaining  together  in 
this  better  final  year  they  didn't  wish  to  "drag  it  all  up  againH  by  being  interviewed.  In  addition,  one  multi- 
media production  company  was  considering  revising  corporate  policy  to  do  educational  media  develop- 
ment entirely  in-house  rather  than  through  partnerships  with  outside  educational  organizations  because  the 
"investment  in  time,  money  and  energy"  involved  in  collaborations  was  deemed  too  taxing  for  the  return* 
Disagreements  between  partners 

The  kind  of  disagreements  most  of  the  other  respondents  denoted  varied  by  the  organizations 
involved,  as  different  concerns  surfaced  among  partnership  representatives  than  between  representatives 
and  teachers  or  representatives  and  technology  vendors.  Among  partnership  representatives  three  areas  of 
disagreement  werc  mentioned  most  often;  allocation  of  resources,  partnership  roles,  and  pedagogy. 

Representatives  often  had  to  choose  among  many  competing  uses  for  the  money,  technology  and 
services  their  partnership  had  to  provide  schools.  There  were  therefore  hard  choices  to  be  made  on  who 
und  what  would  be  funded,  what  hardware  and  software  would  be  chosen,  and  what  direction  stall  lime 
and  energies  would  take.  As  one  project  director  said,  "We  can  think  of  40  things  to  use  the  money  lor  and 
can  only  do  3  oi  4,  so  deciding  what  you're  NOT  going  to  do  with  resources  is  important,..  It  was  hard 
for  us  to  say  no." 

Dissension  also  arose  about  what  role  each  partner  was  to  take  in  the  project.  Often  it  was  not  clear 
what  each  organization  was  expected  to  provide  or  each  representative  to  do,  and  so  representatives  felt 
others  were  not  doing  their  jobs  or,  as  one  project  coordinator  said,  "holding  up  their  part  of  the  bargain." 
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This  lead  to  resentment.  In  addition,  as  projects  evolved,  there  was  often  a  revaluation  ot  partnership 
roles,  some  partners  becoming  more  active  than  others  in  different  stages  of  the  project.  Agreeing  on  who 
should  take  an  active  role  at  any  given  time  also  caused  dissension. 

Last,  partners  did  not  always  agree  on  pedagogy,  which  was  clearest  in  discussions  on  the  proposed 
content  of  educational  materials,  such  as  curricula!*  units,  lesson  plans,  computer  programs  purchased  or 
developed  for  use  in  classrooms,  and  prototype  educational  CD-ROMs  or  videos.  The  director  of  research 
and  development  for  a  national  professional  association  setting  standards  for  the  teaching  of  science 
pointed  to  the  problem  in  his  particular  project.  "There  is  a  wide  range  of  philosophical  differences  on  how 
to  teach  science.. the  clash  of  ideas  sometimes  led  to  belittling  and  ridiculing  of  opposite  positions." 

In  addition  to  these  primary  disagreements,  as  mentioned  earlier,  partners  also  disagreed  to  a  lesser 
extent  about  the  methods  used  to  assess  accomplishment  of  project  goals,  some  wanting  more  "concrete 
benchmarks"  than  others,  and  about  the  project  implementation  process,  particularly  how  to  speed  it  up. 
Disagreements  about  technology 

Because  these  partnerships  all  involved  educational  technology,  respondents  singled  out  one  kind  of 
disagreement  among  partners  for  special  mention.  A  little  over  a  fourth  of  the  disagreements  mentioned 
were  between  other  partners  and  the  technology  vendor  in  partnerships  which  had  major  hardware  and 
software  companies  as  members.  Three  kinds  of  problems  were  mentioned  most  frequently"  slow  de- 
livery of  promised  goods  (hardware  and  software),  unavailability  of  proffered  training,  and  serious 
contention  about  the  content  of  materials  being  collaboratively  developed.  In  the  first  two  cases,  other 
partners  felt  the  service  by  the  technology  partner  "after  the  sale"  was  not  as  anticipated,  cither  because  the 
vendor  primarily  viewed  the  partnership  as  a  chance  to  sell  products  and  services  rather  than  a  true  collab- 
oration or  because,  although  well-intentioned,  many  vendors  were  either  too  busy  to  guarantee  delivery 
schedules  and  training  as  planned,  or  were  too  large  and  bureaucratic  to  secure  the  commitment  of  corp- 
orate headquarters  to  partnership  activities.  The  principal  investigator  on  one  grant  said,  "Over  lime  we  got 
more  (goods  and  training),  but  I'm  not  sure  we  got  more  than  usual  people  who  buy  hardware.  Corporate 
headquarters  of  company  "A"  wouldn't  commit  and  company  "B"  is  always  reorganizing,  so  it  was  too 
little,  too  late  on  gear  and  training." 
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The  problem  over  developing  materials  collaboratively  is  a  good  indication  of  how  different 
organizational  cultures  can  cause  coiillicl.  In  two  cases  serious  differences  arose  between  the  educational 
partners  and  the  media  production  stilt T  of  the  technology  partner  over  the  content  of  educational  materials 
produced.  Naturally  enough,  the  educational  staff  placed  primary  emphasis  on  the  teaching  and  learning 
objectives  of  whatever  was  being  produced  and  the  media  producers  were  primarily  interested  in  its 
interest  level  to  students.  One  grant  principal  investigator  said  working  with  a  scriptwriter  epitomized  this 
conflict  in  approach.  "It  was  hard  to  get  him  to  realize  our  ideas.  We  had  instructional  design  goals,  he  had 
entertainment  goals.  It  was  a  knock-down-drag  out  to  build  both  into  the  script.  The  differences  between 
us  were  very  stark. ..clear  differences  in  conception  which  were  hard  to  blend." 

Two  respondents  also  mentioned  "battles"  between  hardware  and  software  suppliers  trying  to  make 
their  various  technologies  compatible  in  ways  that  would  serve  partnership  goals.  Project  managers  felt 
caught  in  between  two  competing  interests,  unable  to  coordinate  efforts  to  go  forward. 
Disagreements  between  partners  and  teachers 

Two  major  kinds  of  disagreements  arose  between  partnership  representatives  and  teachers.  The 
problem  mentioned  most  often  was  competing  ideas  over  the  amount  and  content  of  training  offeied. 
Teachers  often  preferred  elementary  training  in  how  to  operate  a  computer  and  use  basic  application 
software,  while  partnership  representatives  were  emphasizing  classes  in  integration  of  the  computer  into 
classroom  activities  and  development  of  original  educational  materials.  One  school  district  technology 
coordinator  described  this  as,  "We  were  determined  to  integrate  technology  and  the  teachers  were 
determined  not  to." 

The  other  primary  disagreement  was  over  assessment  and  evaluation  methods.  Project  managers  were 
often  required  by  their  grant  to  regularly  evaluate  progress  and  sometimes  had  concerns  about  teacher 
performance,  so  they  were  interested  in  timely  and  telling  evaluations.  Teachers,  on  the  other  hand,  found 
some  evaluation  procedures  intrusive  or  overly  time-consuming  and  tedious. 

To  summarize,  all  but  four  of  the  partnership  participants  questioned  were  able  to  describe 
disagieements  among  partners.  The  nature  of  the  disagreements  varied  by  who  was  interacting,  but  there 
was  dissension  across  a  broad  range  of  people  and  activities,  including  about  the  most  fundamental  work 
of  the  partnerships  such  as  allocating  money,  assigning  roles,  choosing  among  educational  alternatives, 
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providing  g(x*Js  and  services,  training  staff  and  evaluating  progress.  This  pattern  suggests  that 
partnerships  do  not  escape  the  conflict  which  the  preliminary  analysis  suggested  characterizes  other 
educational  policymaking  toxiies.  Responses  also  suggest  that  the  addition  of  technology  into  a 
partnership  introduces  a  level  of  complexity  which  can  present  unique  challenges. 
Conflict  resolution 

Because  disagreements  can  present  major  ohstaclcs  to  success,  it  is  also  helpful  to  know  how  partners 
tried  to  resolve  conflict  ami  whether  the  methods  they  chose  were  successful.  It's  interesting  to  note,  then, 
that  regardless  of  their  position  or  the  kind  of  conflict  vhey  were  discussing,  respondents  overwhelmingly 
described  their  primary  method  of  resolving  conflict  as  building  "consensus"  by  "talking  it  out". 
Consensus 

Kach  described  a  process  of  dispute  resolution  which  began  with  asking  for  alternatives,  describing 
the  advantages  and  disadvantages  of  each  suggestion  from  each  partner  s  point  of  view,  and  then  choosing 
one  alternative  through  "negotiation"  and  "compromise".  Many  mentioned  they  thought  this  process 
represented  the  essence  of  collaboration--  group  decision-making  through  discussion  and  negotiation.  One 
corporate  education  representative  characterized  it  as  "looking  for  a  compromise.  Seeing  how  we  can 
change  so  that  everybody  enjoys  a  winner.. a  good  partnership  will  give  everybody  something  to  win 
about."  Others  described  it  as  reaching  a  "common  understanding"  or  "tandem  decision -making", 

Most  recognized,  however,  some  limits  to  this  method  of  resolving  conflicts,  l  ust,  it  can  take  a  long 
time,  as  one  project  director  noted,  "Collaborative  work  has  its  own  timeline.  It  is  much  slower  than  when 
telling  them  (others)  what  to  do.  That  isn't  bad. ..it's  just  a  different  pace  which  takes  adapting  to. ' 

In  addition,  many  responses  pointed  to  a  set  of  attitudes  which  must  be  present  for  this  kind  of 
conflict  resolution  to  work.  People  must  trust  each  other,  know  each  other's  working  style,  and  be  willing 
to  critically  examine  and  change  their  own  opinions  and  cx;casionally,  behavior.  The  same  person  con 
!inued,  "You  must  be  willing  to  work  as  much  on  the  relationship  between  people  as  on  tasks.  That  was  a 
personal  learning  for  me  as  a  project  director.  You  must  be  open  to  change  and  be  a  co-learner. "  Another 
project  coordinator  felt  facilitating  group  process  was  the  most  challenging  part  of  her  project,  indicating 
that  most  adults,  even  teachers,  have  no  formal  training  in  collaboration  and  do  not  automatically  know 


14 

■Ud.  138 


how  to  solve  problems  and  make  decisions  together.  The  needed  attitudes  and  actions  were  therefore 
learned,  sometimes  painfully,  in  the  process  of  working  together 
Role  of  the  project  coordinator/manager 

Many  of  the  responses  outlined  a  critical  role  in  the  dispute  resolution  process  for  the  project  director 
or  coordinator,  those  partnership  managers  who  are  responsible  for  the  day-to-day  operation  of  the 
project.  One  said  she  would  only  hire  project  coordinators  like  theirs  who  had  "experience  running 
complex  projects  and  getting  people  to  work  together".  Another  said  a  crucial  part  of  her  role  as  project 
coordinator  was  to  do  the  "summing  up"  at  the  end  of  meetings  which  helped  people  choose  among 
alternatives. 

In  addition,  if  people  were  not  able  to  reach  consensus,  which  many  said  happened  occasionally,  the 
project  coordinator  or  director  often  made  a  final  determination,  sometimes  by  "delaying  a  decision  until  a 
better  time  arose",  or  "chopping  one  idea  and  substituting  another"  or  even  by  "taking  the  project  and 
running  with  it",  i.e.,  making  the  decision  he  or  she  thought  best  for  the  project.  When  this  last  happened, 
the  coordinator  or  director  usually  justified  the  decision  to  the  others  by  reference  to  the  original  purposes 
of  the  project  as  represented  in  project  documents  or  by  citing  grant  demands  made  by  the  government 
funding  agency.  Respondents  clearly  linked  group  decision-making  to  the  idea  ol  a  partnership,  and  felt  it 
"heavy-handed"  when  either  they,  or  someone  else  in  the  partnership,  made  decisions  unilaterally.  As  one 
said,  "It  takes  longer  to  make  decisions  together  but  telling  people  what  to  do  doesn't  work,  so  it  works 
out  better  this  way  in  the  long  run." 

In  three  of  the  23  cases  this  process  of  consensus  didn'i  work  at  all.  Two  ol  these  cases  involved 
school  districts  and  hardware  or  software  companies,  and  one  concerned  a  school  district  and  a  group  of 
teachers.  In  the  first  cases,  the  school  district  partners  found  the  vendors  to  be  uninterested  in  true 
collaboration.  They  claimed  corporations  either  disappeared  after  the  sale  of  hardware,  providing  little 
further  support,  or  overly  limited  school  district  participation  in  joint  activities  with  tightly  worded 
partnership  agreements.  In  one  partnership  between  a  school  district  and  a  multi-media  development 
company,  a  contract  outlining  the  duties  and  obligations  of  each  partner  was  considered  too  restrictive  by 
the  school  district  representative,  who  felt  left  out  of  decision-making  about  the  educational  content  of  the 
product.  Yet  the  same  contract  was  considered  routine  by  the  corporate  representative,  used  to  the  highly 
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competitive  software  industry  in  which  protection  of  corporate  creative  processes  is  paramount  in  any 
collaborative  work. 

This  misunderstanding  points  to  n  clear  difference  of  organizational  culture  as  reflected  in  the 
documents  and  standard  operating  procedures  which  embody  organizational  values.  The  corporate 
representative  explained  school  district  concerns  as  resulting  from  a  scheduling  problem,  "It  was  not 
anyone's  fault.  It  was  just  circumstances.  Theiv  are  differences  in  the  way  schools  and  corporations  carry 
out  their  work  day.  We're  in  a  production  mode  whether  schools  are  in  summer  break  or  not,  so  if  kids 
and  teachers  aren't  around,  I  make  do  with  other  kids,  other  classes." 

She  added  that  the  consensual  process  of  solving  conflicts  such  as  this  was  different  for  her  as  a 
corporate  partner.  "Production  and  creative  decisions  are  made  by  huge  numbers  of  people  bantering  about 
ideas  in  a  software  company.  It  takes  time,  and  feelings  get  hurt.  1  was  relieved  to  work  in  a  situation  (in 
the  schools)  where  people  liked  my  ideas."  She  detailed  a  process  in  which  she  presented  the  partnership 
with  production  company  ideas  about  the  educational  content  of  the  medium  being  produced,  followed  by 
some  discussion  in  which  there  was  minimal  objection  on  minor  issues  ("no  red  flags")  after  which  she 
tried  to  "follow  the  consensus"  in  the  subsequent  production.  She  didn't  feel  she  had  to  "fight  for  my 
ideas"  as  she  would  have  done  in  a  corporate  environment. 

However,  the  school  district  representative  on  the  same  project  felt  the  district  had  only  minimal 
impact  on  the  educational  content  of  the  product.  She  cited  contract  provisions  which  would  financially 
penalize  the  school  district  for  making  any  changes  which  slowed  down  the  production  schedule  which 
she  felt  inhibited  her  group  from  persisting  in  their  pedagogical  concerns.  The  corporate  representative 
apparently  did  not  realize  her  school  district  counterpart  felt  this  strongly,  as  she  thought  there  were  no 
major  disagreements  on  content. 
False  consensus 

This  points  to  a  danger  focused  on  by  another  of  the  respondents-  the  problem  of  acceding  to  a  false 
consensus.  The  desires  to  get  along  amicably,  make  deadlines  and  meet  the  goals  of  the  project  and 
demands  of  the  funding  agency  can  cause  people  to  agree  to  what  they  think  everyone  else  wants  even  if 
they  or  their  organization  feels  what  is  being  proposed  is  a  bad  idea-  a  kind  of  "group  think".  The 
director  of  research  and  development  responsible  for  one  project  referred  to  this  as  the  "Abilene  paradox" 
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after  the  work  of  management  professor  Jerry  Hall,  saying  "a  lot  of  decisions  arc  based  on  people 
agreeing  because  they  think  it  is  what  the  group  wants,  but  no  one  wants  it  really."  To  avoid  this  false 
consensus,  this  director  prefers  open  and  pointed  disagreement  to  acquiescence  in  the  interest  of 
collegiality  or  expedience. 
Exiting' 

The  unresolved  conflict  between  teachers  and  the  school  district  was  over  the  content  of  training  and 
demonstrates  the  success  of  the  "exit"  strategy  in  influencing  group  processes.  School  district  partnership 
representatives,  sensitive  to  the  outlines  of  the  grant  underlying  the  partnership,  wanted  teachers  to 
integrate  technology  into  the  classroom,  bypassing  or  hurrying  through  the  process  of  learning  the  basics 
of  computing  and  common  application  programs  like  word  processing.  But  teachers  did  not  want 
integration  classes  and  chose  not  to  attend  them,  no  matter  how  many  were  offered  by  the  district,  at  what 
times  and  for  what  pay.  They  "exited";  a  powerful  ploy  used  by  those  not  in  power  to  frustrate  the  aims  of 
those  above  them  with  whom  they  don't  agree.  This  conflict  was  never  resolved  to  the  district's 
satisfaction,  and  this  part  of  their  project  went  unrealized  as  a  result. 

"Exiting"  was  also  used  as  a  strategy  by  partnership  representatives  in  many  of  the  other  projects, 
even  where  consensus  did  work  to  solve  most  issues.  At  least  five  respondents  mentioned  partners  who 
had  withdrawn,  usually  the  corporate  partners,  individual  school  districts  or  individual  schools.  Four 
others  mentioned  changes  in  personnel,  moving  partnership  representatives  around  until  they  found  a 
"stable  working  group"  which  could  "deliver  on  what  they  promised."  Indeed,  as  mentioned  earlier,  high 
turnover  in  personnel  was  cited  in  three  cases  as  a  major  obstacle  to  accomplishment  of  project  goals,  as 
new  people  had  to  be  "brought  up  to  speed"  before  the  partnership  could  proceed. 

Four  partnership  representatives  also  mentioned  threatening  to  exit,  i.e..  stating  their  intent  to 
withdraw  partnership  resources  from  school  districts,  schools  or  teachers  who  were  not  fulfilling  their 
obligations.  These  "threats"  were  often  first  carefully  stated  in  writing  with  follow-up  telephone  calls  or 
site  visits  by  project  directors  or  coordinators.  Usually  such  threats  were  effective.  Where  partners  or 
participants  withdrew  it  was  of  their  own  accord,  few  were  "dropped"  by  the  others. 

In  summary,  most  partnerships  resolve  conflicts  by  identifying  and  weighing  alternatives  and  talking 
things  through  to  a  consensus,  a  time-consuming  process  of  negotiation  and  compromise  which  partners 
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recognize  as  requiring  special  skills  in  group  process,  and  a  willingness  by  partners  to  learn  new  things, 
be  open,  trust  each  other,  and  change  positions, 

In  addition,  project  coordinators  or  directors,  as  the  day-to-day  managers  of  most  technology 
partnerships,  frequently  have  a  special  role  to  play  in  conflict  resolution  as  the  ones  who  make  partners 
confront  each  other  and  the  issues,  who  summarize  positions  so  decisions  can  be  made,  and  who  take  over 
decision-making  when  an  impasse  is  reached.  They  are  often  the  facilitators  of  conflict  resolution  and  the 
guardians  of  group  process.  It  is  seen  as  part  of  their  job  by  the  others  involved. 

Also,  where  consensus  fails  to  resolve  disputes,  the  "exit"  strategy  is  often  used  by  partnership 
representatives,  who  either  withdraw  or  threaten  to  withdraw  their  organizational  resources  from  the 
partnership.  Exiting  can  be  represented  formally  by  threats  to  retract  manifested  in  documents,  phone  calls 
or  visits,  or  informally  by  non-participation  in  meetings  or  activities,  i.e.,  partners  can  leave  or  they  can 
fade  away.  They  are  seldom  booted  out. 

Last,  beware  of  false  consensus.  The  desire  to  please  each  other  and  forge  ahead  can  lead  people  to 
agree  to  what  they  don't  really  like  or  think  practicable,  which  can  create  problems  when  differing 
organizational  cultures  and  standard  operating  procedures  undermine  underexamined  goals. 

What  features  are  identified  as  successful  most  often?  What  advice  do  participants 
have  for  others  beginning  partnerships? 

Answers  to  these  research  questions  came  from  reponses  to  three  questions  on  the  questionnaire 
including,  What  features  of  their  partnership  lead  to  success?,  What  would  you  change  if  you  could  begin 
again?  and  What  advice  do  you  have  for  those  beginning  partnerships?  Answers  to  the  first  and  third  ^ 
questions  overlapped  significantly,  so  they  will  be  discussed  together  below. 

When  respondents  were  asked  what  they  would  change  about  their  partnerships  if  they  could  start 
over,  8  of  23,  almost  a  third,  said  nothing.  Half  of  these  didn't  identify  any  mapx  problems  in  the  course 
of  their  partnerships,  and  the  other  half  didn't  feel  the  problems  they  experienced  were  debilitating  enough 
to  warrant  making  any  changes.  Overall  these  representatives  were  fairly  satisfied  with  the  way  things 
went. 


,  t.  18 

^  142 


Choose  people  and  sites  carefully 

The  other  two-thirds  would  change  their  partnerships  in  a  variety  of  ways,  most  frequently  by 
including  either  different  people  or  a  wider  range  of  people.  Eleven  references  were  made  to  being 
'choosier'  in  who  the  partnership  included.  These  were  usually  made  by  representatives  of  the  imitating 
partner,  the  organization  which  first  solicited  other  partners  and  drafted  the  grant  proposal.  These 
respondents  felt  the  push  to  find  sites  and  encourage  participation  sometimes  brought  people  and 
organizations  together  without  due  consideration  for  their  true  willingness  or  ability  to  actively  participate. 
Several  representatives  mentioned  the  importance  of  being  able  to  pick  who  they  worked  with,  whether 
that  was  someone  in  a  partner  organization  with  whom  they  "already  had  a  working  relationship",  some- 
one who  could  demonstrate  their  ability  to  "deliver  what  promised"  or  someone  who  chose  to  be  included 
rather  than  being  assigned  by  administrative  staff. 

Respondents  would  also  be  more  selective  in  picking  project  sites,  choosing  fewer  districts  or 
schools,  sites  geographically  closer  to  the  project  administration  point,  or  those  which  represented  "ideal 
conditions".  Two  even  mentioned  having  some  kind  of  competitive  process  for  inclusion  in  the  project. 
One  project  director  who  had  worked  with  a  large  number  of  schools  and  districts  over  the  three  years  of 
her  project  said,  "I  would  be  choosier..,look  for  optimal  conditions  rather  than  anyone  who  wanted  to 
work,  and  put  conditions  on  schools  to  be  selected.  Have  them  put  in  writing  what  they  will  do  to  support 
us." 

Eight  references  were  made  to  including  a  wider  range  of  people,  usually  to  ensure  critical  support 
from  interested  "stakeholders",  particularly  principals.  As  the  people  primarily  responsible  for  decision- 
making and  allocation  of  resources  in  school  buildings,  principals  are  crucial  to  the  success  of  any 
educational  change  effort,  technology  projects  included  One  project  director  said  about  administrative 
support  for  her  teacher  training  project,  "There  should  be  some  really.  For  the  most  part,  they  didn't 
support.  There  wasn't  an  administrator  at  the  schools  who  knew  and  who  cared.  It  made  it  hard  for 
teachers  to  request  individual  support."  One  school  district  technology  director  who  had  helped  choose 
sites  for  a  project  by  mailing  a  survey  to  the  schools  added,  "Now  I'd  go  to  individual  schools  and  inter- 
view principals  and  technology  teachers...I'd  do  more  personal  interfacing  with  principals.  It  was  more 
important  than  I  realized." 
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Some  answers  to  Question  8,  (Do  you  have  any  advice  for  those  starting  partnerships  now  which  will 
help  them  be  successful?)  corroborated  this  emphasis  on  including  all  important  stakeholders.  Three 
references  were  made  to  soliciting  comments  from  impacted  groups  during  the  grant  writing  and  planning 
stages  to  secure  their  ideas  and  interest  early,  and  others  mentioned  including  everyone  at  all  levels  in 
every  phase  of  the  project  (planning,  implementation,  evaluation)  and  securing  organizational,  rather  than 
merely  personal,  buy-in.  This  last  suggestion  reflects  the  experience  of  projects  where  a  critical  person  or 
persons  exited  during  the  project,  taking  their  organizational  commitment  with  them.  Representatives  who 
had  experienced  this  problem  suggested  getting  an  institutional  commitment  in  writing  before  the  project 
begins.  Others  mentioned  "keeping  the  original  personnel  if  possible". 
Technology 

The  remaining  answers  to  the  question  about  what  partnership  representatives  would  change  were  all 
specific  to  improving  the  technical  aspects  of  the  project,  but  there  was  no  clear  pattern  to  the  responses. 
Instead  each  responded  about  the  particulars  of  their  own  project.  However,  several  mentioned  an  im- 
portant point  which  was  assumed,  though  not  stated,  in  most  responses-technical  stability  and  reliability. 
In  every  project  technology  was  seen  as  the  means  to  a  desired  end,  usually  some  improvement  in  teaching 
and  learning,  an  end  which  could  not  be  accomplished  if  the  critical  technology  needed  was  absent, 
broken,  unreliable  or  overly  complex.  Having  dependable  technology  is  critical  to  these  kinds  of  projects. 

The  largest  set  of  responses  beyond  this  basic  point  involved  improving  teacher  training,  suggesting 
that  training  is  viewed  as  critical  to  the  success  of  many  of  the  projects  but  can  be  hard  to  do  effectively. 
Suggestions  here  were  varied,  including  "begin  with  teachers  beyond  the  basic  learning  level",  "ask  for 
one  year  to  train",  "use  technology  mentor'  teachers  for  peer  training",  and  "monitor  the  quality  of  in- 
service  training". 

Respondents  also  made  a  group  of  suggestions  about  production  of  educational  materials,  including 
"get  a  production  staff  with  educational  experience",  "use  existing  materials  rather  than  try  to  develop  your 
own  t  "leave  plenty  of  time  for  production",  "teach  the  government  [DOE]  to  be  flexible  in  production 
schedules"  and  most  interestingly,  "be  entrepreneurial.. .use  the  government  seed  money  to  form  a 
partnership  with  a  production  house  to  develop  educational  materials  and  use  the  profits  from  the  product 
to  fund  district  technology  needs  and  projects." 
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Have  clear  roles  and  expectations 

When  giving  advice  to  those  starting  partnerships  (Question  8)  respondents  overwhelmingly  pointed 
to  the  necessity  of  outlining  clear,  specific  roles  and  responsibilities  for  each  partner  which  are  stated  in 
writing  and  agreed  to  before  the  project  begins.  Suggestions  for  inclusion  in  these  kind  of  written  agree- 
ments were  a  "few  clear,  simple  and  direct  statements  of  project  goals",  a  "formal  team  structure",  "de- 
fined outcomes",  and  a  timeline  for  accomplishment  of  project  "benchmarks".  The  purposes  of  such 
written  agreements  were  stated  variously  as  to  "make  expectations  clear",  "assign  responsibilities",  allow 
for  consideration  of  each  partner's  "real  constraints",  get  to  "know  each  other's  strengths"  and  make  sure 
the  team  is  "moving  in  the  same  direction,  not  on  individual  agendas."  One  corporate  education  manager 
described  it  as  setting  up  a  "win/win  situation"  from  the  beginning,  and  advised  re-negotiating  the  agree- 
ment every  year  to  adapt  it  to  current  conditions,  making  it  a  "living  document". 
Cultivate  collaborative  skills 

Another  large  group  of  advice  responses  centered  on  cultivating  the  personal  characteristics  that  col- 
laboration requires.  Partnership  representatives  suggested  involving  or  developing  people  who  were 
flexible,  open,  creative,  willing  to  "share  ideas"  and  capable  of  working  on  a  basis  of  trust  and  respect. 
They  should  also  be  daring,  "willing  to  push  the  limits  of  what  is  possible"  .  In  addition,  four  references 
were  made  to  the  need  to  be  dedicated  and  hardworking,  i.e.,  people  for  whom  this  is  not  just  a  job,  but  a 
passion.  The  primary  reason  for  the  latter  suggestion  is  because  of  the  huge  amount  of  time  collaborative 
partnership  work  requires,  often  far  beyond  what  people  feel  they  are  paid  for  or  what  their  institutions 
anticipated  when  assigning  them. 
Pay  attention  to  group  process 

The  last  major  group  of  suggestions  for  those  starting  partnerships  had  to  do  with  group  process  and 
dynamics.  Here  the  most  frequent  suggestion  was  to  meet  often  and  communicate  regularly  so  that  people 
feel  included  and  know  what's  going  on.  Several  representatives  also  mentioned  picking  a  good  project 
coordinator  or  director,  someone  who  could  prove  a  strong  'anchor'  for  the  project.  And  others  advised 
openly  recognizing  outside  constraints  on  the  group,  such  as  limits  on  time,  staff  and  resources,  "systemic 
influences"  such  as  school  district  politics,  and  the  impact  of  cultural  differences  between  organizing 
institutions.  One  school  district  technology  coordinator  said,  "You  must  understand  and  accept  the  culture 
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differences.  If  you  don't,  then  there  will  be  continual  problems.  For  example,  university  people  must  get 
something  publishable  out  of  an  experience-  but  teachers  couldn't  care  less  about  that.  The  need  to 
publish  has  implications  for  how  you  spend  time.,,  this  must  be  understood  and  acknowledged." 

Tb  summarize,  the  primary  advice  of  this  seasoned  group  of  partnership  representatives  was  to  be 
specific  and  clear  about  the  goals  of  the  partnership  and  to  clearly  define  in  writing  before  the  project 
begins  what  each  partner  will  be  expected  to  do  to  advance  those  goals.  They  also  strongly  recommended 
pr  motion  of  the  personal  values  and  characteristics  which  make  collaboration  easier-  qualities  of 
flexibility,  openness,  trust  and  respect  for  others  mixed  withcreativity,  daring  and  dedication.  Last, 
respondents  thought  it  wise  to  pay  attention  to  group  process  by  meeting  regularly,  talking  about  things, 
relying  on  the  guidance  of  a  strong  project  coordinator  or  director,  and  acknowledging  systemic  limits  on 
what  can  be  accomplished. 

Are  there  differences  between  partners  in  what  features  are  seen  as  successful  or 
problematic? 

Answering  this  question  for  the  group  of  respondents  as  a  whole  proved  difficult,  largely  because 
corporations  were  underrcpresented  among  those  interviewed.  (5  of  23  respondents).  Where  more  than 
one  person  was  interviewed  from  a  project  (8  of  23  cases),  the  responses  of  each  were  compared  to  each 
other  to  look  for  differences.  Many  of  those  differences  have  been  discussed  in  earlier  analysis.  However, 
more  definitive  conclusions  would  be  possible  with  a  larger  corporate  sample. 

It  is  interesting  to  note,  however,  why  businesses  are  not  more  represented.  Business  partners  were 
the  hardest  to  find  to  interview,  in  many  cases  because  either  their  personal,  or  their  institutional,  commit- 
ment was  relatively  short-term  and  they  were  no  longer  active,  working  members  of  the  partnership.  In  the 
special  case  of  technology  vendors  as  partners,  corporate  involvement  often  ceased  or  was  sharply  cur- 
tailed shortly  after  the  original  purchase  or  donation  of  hardware  and  software.  Businesses  which  had 
long-term,  ongoing  commitments  to  a  partnership  were  rare,  even  though  as  one  corporate  education 
manager  avowed,  "it's  the  only  way  to  be  truly  effective.., roll  up  your  sleeves  and  get  involved  in  every 
aspect." 
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Implications 

It  is  clear  that  partnerships  are  a  growing  phenomenon,  pan  of  the  ever-increasing  overlap  between 
schools  and  other  major  community  institutions,  particularly  businesses  and  universities.  People  who 
involve  themselves  in  partnerships  generally  enjoy  their  participation,  and  have  high  hopes  their 
collaboration  will  make  education  better,  thereby  profiting  society  generally. 

These  high  hopes,  however,  can  mean  partners  do  not  anticipate  many  of  the  problems  which 
managing  a  complex  project  can  bring,  the  conflict  which  can  result  when  organizations  of  disparate  nature 
try  to  work  together,  and  the  time  and  energy  it  can  take  to  work  it  all  out  successfully.  Partnerships  are 
not  the  panacea  to  educational  ills  some  have  suggested.  They  are  hard  work-  and  hard  work  usually 
added  to  the  already  busy  lives  of  many  of  the  professionals  involved.  This  is  all  the  more  true  with 
technology  partnerships,  which  often  add  a  whole  level  of  complexity  to  already  complicated  situations. 

Yet  the  needs  partnerships  rose  to  fill  aren't  likely  to  go  away  soon,  and  the  trend  in  education,  and 
perhaps  society  in  general,  is  increasingly  toward  disparate  groups  of  people  finding  ways  to  work  to- 
gether  for  their  mutual  benefit.  One  project  coordinator  summed  it  well,  reacting  with  excitement  to  her 
role  as  a  "bridge"  between  the  research,  corporate  and  education  communities,  a  "link"  which,  even  if 
temporary,  greatly  enriches  the  experience  and  understanding  of  everyone  involved.  This  "bridging"  role 
is  still  developing,  and  as  such  is  fraught  with  peril  for  those  involved,  but  it  also  brings  the  stimulation  of 
challenge  and  possibility. 
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Appendix  I.  Interview  Protocol 


Name  of  respondent:  

Title  of  respondent:  

Respondent  contact  numbers : 

phone: '  e-mail: 

Project  Title  :  


Dpscrintive  Questions 

1.  Who  are  the  partners? 

a.  Organizational  Name: 
Organizational  Type:  

b.  Organizational  Name: 
Organizational  Type:  

c.  Organizational  Name: 
Organizational  Type:  

d.  Organizational  Name: 
Organizational  Type:  


2.  What  role(s)  was  each  partner  originally  designed  to  play? 

Organization  a:  


Organization  b: 


Organization  c: 


Organization  d: 


3.  Who  initiated  the  partnership? 


Name  of  Interviewer:  

Permission  to  quote?:  yes 
no   
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Description  of  Partnership  Operations 


How  does  your  project  coordinate  partnership  activities? 

a)  Do  you  have  regular  meetings?      yes   no_ 

If  yes, 

how  often?  

where?  


who  attends  ?  

is  there  an  agenda?    yes  no 

If  yes,  who  sets  the  agenda? 


If  yes,  how  are  items  chosen  for  discussion? 


If  no,  how  is  the  discussion  organized? 


If  no,  (your  partnership  does  NOT  have  formal  meetings), 

how  do  partners  decide  what  needs  doing  and  how  to  do  it? 


b)  How  else  do  partners  coordinate  their  activities? 
telephone? 


correspondence?  (notes,  e-mail,  letters,  memos,  fax  etc.) 

(probe)  any  other  ways  you  can  think  of? 

c)  who  makes  most  of  the  day-to-day  project  decisions? 
why  this  person(s)? 
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Partnership  Affect 

I.  What  do  you  like  about  being  pan  of  a  partnership  or  collaboration? 


(probe)  anything  else  you  like? 

2.  What  do  you  dislike  about  being  part  of  a  partnership/collaboration? 


(probe)  anything  else  you  dislike? 


3.  when  disageements  about  partnership  activities  arise,  how  are  they  resolved?  (Give  example?) 


(probe) 

any  other  ways  of  resolving  disagreements? 


4.  on  which  partnership  activities  are  partners  most  likely  to  disagree? 


(probe)  any  other  disagreements  you  can  think  of? 
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Perceptions  of  Success 

I.  What  was  the  original  role  of  educational  technology  in  the  project? 


2.  How  central  was  that  role  to  the  entire  scope/activities  of  the  project? 


2.  Has  the  role  of  technology  changed  as  the  project  progressed?  yes  no 

If  yes,  how  has  it  changed? 


why  did  it  change? 


3.  Do  you  feel  the  partnership  has  been  successful  in  helping  attain  the  project's  technical  goals? 

yes  no  

If  yes,  what  features  of  the  partnership  helped  technology  succeed? 


(probe)    Any  other  ways  the  partnership  helped  support  technical  goals? 


If  no,  what  features  of  the  partnership  impeded  success? 


(probe)  Any  other  aspects  of  the  partnership  which  impeded  success? 
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4.  How  does  the  partnership  gauge  success? 

Are  there  concrete  products? 
If  yes,  what  are  they? 


yes 


If  no,  how  else  is  success  gauged? 


(probe)   any  other  ways  the  partnership  gauges  success? 


5.  Do  you  personally  agree  with  these  ways  of  gauging  success?  yes 
Why  or  why  not? 


5.  If  you  could  begin  again  ,  what  would  you  change  about  the  partnership? 


(probe)  is  there  anything  else  you  would  change? 


6.  Do  you  have  any  advice  for  people  forming  partnerships  now  that  will  help  them  be  successful? 


(probe)  any  other  advice? 
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7.  Would  you  predict  the  partnership  will  continue  after  the  grant  is  continued?  yes 
If  yes,  where  will  the  money  come  from? 


Have  you  submitted  other  grants?    yes   no 

If  no,  why  not? 


7.  Has  your  project  been  formally  evaluated?  yes  no 

If  yes,  internally?  externally?  both? 

Would  you  be  willing  to  share  those  evaluations  with  us?  yes  no 

8.  Would  you  like  a  copy  of  the  results  of  this  survey  when  it  is  completed?  yes  _ 

If  yes,  send  to  :  


9.  Do  you  have  anything  you'd  like  to  add? 
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Using  and  Selecting  Graphic  Techniques  to  Acquire  Structural  Knowledge 

Abstract 

Structural  knowledge,  the  knowledge  of  relationships  between  concepts  in  a 
content  area,  is  essential  for  comprehension  and  problem  solving.  One  method  of 
assisting  learner  in  acquiring  structural  knowledge  is  through  the  use  of  graphic 
techniques  as  learning  strategies.  Graphic  techniques,  such  as  networks,  pattern 
notes,  semantic  maps  and  graphic  organizers,  are  two-dimensional  diagrams  of 
conceptual  relationships.  In  this  paper  the  characteristics  of  graphic  techniques  are 
described,  and  the  cognitive  processes  used  to  construct  the  diagrams  are  proposed. 
We  propose  a  model  for  selecting  between  different  graphic  techniques  based  upon 
the  cognitive  processes  elicited  by  the  technique,  and  the  learning  outcomes  derived 
from  these  processes. 
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Structural  knowledge  is  knowledge  that  represents  the  relationships  between 
concepts  in  a  content  domain.  While  structural  knowledge  may  be  conveyed  in  a 
number  of  ways,  recently  there  has  been  an  increased  interest  in  techniques  that 
display  structural  knowledge  through  graphic  depictions  of  the  relationships 
between  concepts.  This  paper  is  based  upon  the  premise  that  when  different  graphic 
techniques  are  used  as  learning  tools  different  cognitive  processes  are  elicited, 
resulting  in  different  learning  outcomes.  The  purpose  of  this  paper  is  twofold. 
First  we  describe  the  value  of  conveying  structural  knowledge  and  the  use  of 
graphic  techniques  to  convey  this  knowledge.  Second,  we  present  a  model  which 
describes  differential  learning  outcomes  of  a  variety  of  graphic  techniques  based 
upon  the  technique's  attributes  and  the  types  of  cognitive  processes  that  are  elicited 
when  the  graphics  are  constructed. 

Structural  Knowledge 
Structural  knowledge  is  an  important  construct  in  learning  and  instruction  for 
several  reasons:  1)  structure  is  inherent  in  all  knowledge,  2)  structural  knowledge  is 
essential  to  comprehension,  3)  learners  acquire  structural  knowledge  as  a  natural 
outcome  of  instruction,  4)  experts  represent  structural  knowledge  differently  from  novices, 
and  5)  structural  knowledge  is  essential  to  problem  solving. 

1)  Structure  is  inherent  in  all  knowledge.  "Meaning  does  not  exist  until  some 
structure,  or  organization,  is  achieved'  (Mandler,  1983,  p.  4).  Without  structure,  mental 
constructs  could  not  be  formed,  because  nothing  could  be  described.  The  more  abstract  the 
ideas  are,  the  more  important  structure  becomes. 

2)  Therefore,  structural  knowledge  is  essential  to  comprehension.  We  naturally  and 
necessarily  organize  our  mental  representations  of  phenomena  in  order  to  be  able  to  access 
them.  Structural  knowledge  has  greater  importance  as  the  task  increases  in  complexity. 
For  instance,  meaning  for  text  evolves  from  story  schema*  (structures)  that  readers 
construct  by  integrating  them  with  exisiting  knowledge  structures.  Bruner  (1960)  believed 
in  the  need  to  teach  students  how  ideas  or  concepts  within  a  content  area  are  linked, 
arguing  that  failure  to  learn  this  structural  information  results  in  an  inability  to 
comprehend  ideas  and  transfer  them  to  new  situations.  Acquiring  knowledge  about  the 
structure  of  a  content  area  allows  learners  to  better  comprehend  and  retain  content  and 
apply  it  to  new  situations.  Bruner  believed  that  "knowledge  one  has  acquired  without 
sufficient  structure  to  tie  it  together  is  knowledge  that  is  likely  to  be  forgotten"  (Bruner, 
1960,  p  31). 

3)  Learners  acquire  structural  knowledge  as  a  natural  outcome  of  instruction.  As 
instruction  progresses,  the  learners'  conceptions  of  content  (including  structure)  become 
more  interrelated  and  correspond  more  closely  with  the  teacher's  content  structure 
(Shavelson,  1974).  So  learners  acquire  structural  knowledge,  as  well  as  declarative  and 
procedural  knowledge,  from  instruction. 

4)  Experts  represent  structural  knowledge  more  efficiently  and  effectively  than 
novices.  Among  the  major  differences  between  experts  and  novices  is  that  experts' 
knowledge  includes  rich  sets  of  pattern-indexed  schemata  that  guide  problem 
interpretation  and  solution  (Larkin,  McDermott,  Simon,  &  Simon,  1980).  These  schemata 
are  the  expert's  structural  knowledge  about  his/her  field.  Experts  develop  more  elaborate 
schemata  and  these  richer  schemata  involve  more  complex  structures  that  enable  them  to 
reason  more  effectively. 

5)  Structural  knowledge  is  essential  to  problem  solving.  A  number  of  research  studies 
have  demonstrated  the  importance  of  structural  knowledge  to  problem  solving  (Chi  & 
Glaser,  1985).  The  extent  to  which  problem  solving  protocols  contained  relevant  structural 
knowledge  is  the  strongest  predictor  of  how  well  learners  solve  transfer  problems  in 
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physics  (Robertson,  1990).  Domain-rpecific  problem  solving  relies  on  adequate  and 
appropriate  knowledge  structures. 

The  acquisition  of  structural  knowledge  is  integral  to  recall,  comprehension,  and 
transfer.  Structural  knowledge  provides  a  semantic  foundation  for  the  comprehension  of 
materials  and  solution  of  domain-specific  problems.  Therefore,  it  is  important  to 
facilitate  the  construction  and  acquisition  of  structural  knowledge  during  learning.  One 
approach  to  facilitating  structural  knowledge  acquisition  is  the  explicit  elicitation  and 
communication  of  structural  knowledge  during  learning  and  instruction.  That  is, 
knowledge  structures  are  elicited  from  experts  or  experienced  practitioners  and  mapped 
onto  instructional  materials.  This  mapping  may  be  implicitly  conveyed  in  the 
organization  of  the  content  or  graphically  depicted  for  the  learner. 


Graphic  Techniques 
Graphic  techniques  are  spatial  representations  of  structural  knowledge  in  a 
content  area.  As  noted  above,  these  representations  can  be  developed  by  experts  and 
provided  to  learners  to  promote  acquisition  of  structural  knowledge.  However,  the 
purpose  of  this  article  is  to  examine  the  use  of  graphic  techniques  as  learning 
strategies  drawn  by  learners  as  they  study  material  from  texts,  lectures,  films  and 
other  media.  Although  each  of  the  techniques  can  be  used  to  depict  structural 
knowledge,  they  differ  from  each  other  in  appearance,  types  of  relationships 
displayed,  and  the  use  and  type  of  labels  used  to  name  concept  relationships. 

Appearance 

The  most  obvious  difference  between  graphic  techniques  is  the  format  and 
appearance  of  the  end  product.  Some  types  of  graphics  use  a  matrix  format,  others 
have  lines  connecting  concept  words,  and  another  uses  box-like  frames  sectioned 
into  further  squares  and  rectangles.  The  actual  appearance  of  these  graphics  has 
little  to  do  with  their  effectiveness  in  representing  structural  knowledge;  however, 
those  strategies  with  more  complex  appearances  may  be  more  difficult  to  learn  or  to 
interpret. 

Types  of  Relationships 

The  number  and  types  of  relationships  that  are  depicted  by  each  technique  also 
differs.  Some  convey  hierarchical  information  in  which  concepts  are  subsumed  by 
broader,  more  inclusive  concepts.  While  this  hierarchical  information  is  a 
component  of  content  structure,  it  depicts  only  a  limited  representatio  n  of  that 
structure.  Other  spatial  strategies  permit  the  representation  of  a  broader  variety  of 
relationships  and  interrelationships  between  concepts.   This  depiction  of  multiple 
interrelationships  between  concepts  is  referred  to  as  a  heterarchical  structure. 

Labelling 

Some  graphic  techniques  use  no  labels  so  that  the  relationships  between  concepts 
are  not  clearly  defined.  While  the  learner  may  mentally  identify  the  type  of 
relationship  between  concepts  when  developing  the  graphic  representation,  these 
relationships  are  not  explicitly  identified  on  the  product.  When  multiple 
relationships  arc  depicted  confusion  regarding  the  type  of  relationship  between 
concepts  may  arise  if  no  labels  are  used.  The  strategies  that  use  labels  to  identify 
the  type  of  relationship  that  exists  between  concepts  may  allow  the  learner  to  create 
their  own  labels,  or  use  a  predefined  set  of  relationship  types. 

The  type  of  labelling  system  has  some  impact  upon  the  usefulness  of  the 
technique.  If  one  attempts  to  represent  a  content  area  with  an  insufficiently 
inclusive  labelling  system,  misinterpretations  of  the  final  product  may  result  On 
the  other  hand,  those  strategies  which  use  prespecified  labels  require  the  person 
drawing  the  map  to  identify  not  only  the  concepts,  but  also  the  relatior.ships  between 
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those  concepts.  An  additional  analysis  is  necessary  to  fit  the  relationship  label  into 
a  prespecified  taxonomy.  There  is  some  evidence  that  this  extra  processing 
requirement  may  be  beneficial  to  learners.  Holley  and  Dansereau  (1984)  reported 
that  when  a  modified  form  of  networking  using  learner-generated  labels  was  field 
tested,  learners  with  high  verbal  ability  benefitted  from  the  networking  strategy, 
but  learners  with  low  verbal  ability  did  not  Low  verbal  ability  learners  showed 
greater  retention  and  recall  of  content  when  they  used  prespecified  labels  of 
relationships. 

While  the  appearance  of  the  graphic  and  the  use  of  labels  to  identify 
relationship  type  may  impact  learning,  the  most  important  difference  between 
representational  strategies  appears  to  be  the  types  of  relationships  depicted.  More 
specifically,  it  seems  that  the  primary  difference  in  learning  outcomes  is 
dependent  upon  whether  the  representations  are  strictly  hierarchical  or  if  they 
allow  for  depiction  of  multiple  (heterarchical)  relationships. 

Graphic  techniques  used  as  learning  tools  include  semantic  maps,  semantic 
features  analysis,  spider  maps,  structured  overviews,  pattern  notes,  concept  maps, 
networks,  text  maps  and  somatizations.  In  the  following  section  each  of  these 
representations  arc  described  in  terms  of  the  three  attributes  described  above. 

Types  of  Graphic  Learning  Tools 

Graphic  techniques  that  utilize  a  hierarchical  structure  vary  in  their  visual 
format-using  linear,  matrices,  or  web-like  representations.  The  structured 
overview  (later  renamed  graphic  organizer)  uses  a  linear  representation  in  which 
the  most  important  topic  of  a  passage  or  a  lesson  is  listed  at  the  top,  with  important 
sub-topics  presented  underneath,  and  further  sub-topics  for  each  of  them  listed  next 
(Barron  1969,  1980).  The  organizer  continues  until  all  of  the  important  ideas  are 
included  on  the  graphic.  Thus,  hierarchical  relationships  between  terms  are 
shown  through  the  use  of  unlabeled  lines  (see  Figure  1  for  example). 


Insert  Figure  1  about  here 


The  semantic  features  analysis  is  another  hierarchical  graphic  technique  chat 
that  utilizes  a  matrix  rather  than  lines  to  depict  relationships  (Johnson,  Toms- 
Bronowski,  and  Pittleman,  1982;  Johnson,  Pittleman,  Toms-Bronowski,  Levin, 
1984).  In  order  to  perform  this  technique,  the  general  topic  of  instruction  is  selected, 
and  words  related  to  the  topic  are  identified  and  listed  in  a  column  (see  Figure  2  for 
example).  Then  features  shared  by  some  or  all  of  the  words  are  listed  in  a  row 
across  the  top,  forming  a  matrix.  Next,  learners  indicate  whether  each  concept  has 
each  of  the  features  by  putting  a  "+"  or  a  "-"  in  each  of  the  boxes  corresponding  to  the 
concept  and  the  feature. 


Insert  Figure  2  about  here 


Semantic  maps  and  spider  maps  use  a  web-like  format  rather  than  a  linear 
structure.  Semantic  mapping  (Johnson  et  al,  1982;  Pittleman,  Levin,  and  Johnson, 
1985;  Johnson  et  al,  1984;  Heimlich  and  Pittleman, 1986)  is  relatively  simple.  A 
general  topic  is  defined  and  written  in  the  center  of  a  piece  of  paper  (or  chalkboard). 
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Then  concepts  that  are  related  to  the  main  topic  are  identified  and  connected 
around  the  main  topic  in  a  web-like  fashion  with  unlabelled  lines.  Examples  of 
these  secondary  concepts  are  identified  and  simply  listed  beneath  these  concepts. 
Thus,  a  three-tiered  hierarchy  of  concepts  is  generated  and  graphically 
represented.  (See  Figure  3  for  example). 


Insert  Figure  3  about  here 


A  Spider  Map  is  a  graphic  technique  developed  by  Hanf  (1971)  as  an 
alternative  to  traditional  notetaking  from  text.  In  spider  mapping,  the  main  idea 
of  the  text  passage  is  written  in  the  center  of  a  page,  and  related,  subordinate 
concepts  are  drawn  on  lines  connected  to  the  central  idea  (see  Figure  4  for 
example).  Additional  lines  with  increasingly  detailed  content  can  be  added  to  the 
drawing,  with  the  end  product  resembling  a  spider  Web. 


Insert  Figure  4  about  here 


The  next  group  of  spatial  techniques  utilizes  interrelational,  heterarchical 
representations.  Examples  of  this  technique  include  pattern  notes,  concept  maps, 
semantic  networks,  schematization,  and  text  maps. 

Pattern  notes,  a  technique  very  similar  to  spider  mapping,  was  developed 
originally  as  brain  patterns  by  Buzan  (1974)  and  subsequently  retermed  pattern 
notes  (Fields,  1982;  Jonassen,  1984,  1987)  (See  example  in  Figure  5).  Like  spider 
maps,  pattern  notes  are  drawn  in  a  free-flow,  unconstrained  manner.  The  pattern 
note  is  created  by  writing  the  main  topic  on  the  center  of  a  page,  with  a  box  drawn 
around  it  Then  related  ideas  are  drawn  on  lines  connected  to  the  center  box. 
Additional  lines  and  ideas  are  added,  forming  multiple  layers  arourd  the  central 
idea  until  all  of  the  important  ideas  about  the  topic  have  been  included  on  the  note. 
A  final  step  in  the  pattern  noting  process,  which  distinguishes  this  technique  from 
spider  mapping,  is  to  draw  additional  lines  linking  ideas  to  show 
interrelationships  between  concepts. 


Insert  Figure  5  about  here 


A  modification  of  pattern  notes  was  developed  by  Jonassen  (1984)  to  allow  for 
labelling  of  relationships  between  concepts.  Without  labels,  pattern  notes  can  be 
difficult  to  interpret,  as  one  cannot  be  certain  of  the  type  of  relationship  represented 
by  the  connecting  lines.  Thus,  strategies  that  include  labels  of  the  relationships 
can  be  helpful  in  clarifying  content  structure.  Labelled  pattern  notes  use 
predefined  symbols  to  depict  the  relationships  that  exist  between  concepts. 

Concept  maps  (Novak  and  Gowin,  1984,  Novak,  1980,  1981;  Stewart,  Van  Kirk 
and  Rowell,  1979;  Stewart,  1984)  consist  of  concepts,  or  "nodes"  linked  by  labeled 
lines  to  show  relationships  and  interrelationships  between  terms.  Unlike  pattern 
noting,  in  concept  maps  the  concepts  are  arranged  hierarchically,  so  that  the  most 
inclusive,  subsumptive  concepts  appear  at  the  toy  of  the  map,  with  less  inclusive, 
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subordinate  concepts  below  (see  example  in  Figure  6).  Like  pattern  notes, 
heterarchical  relationships  between  concepts  are  depicted  by  lines.  Here,  labels  of 
the  relationship  type  are  simply  words  selected  by  the  learner,  rather  than  symbolic 
labels  representing  a  fixed  set  of  relationship  types. 


Insert  Figure  6  about  here 


Networking  (Dansereau,  1978;  Dansereau,  Collins,  McDonald,  Holley, 
Garland,  Diekhoflfand  Evans,  1979;  Dansereau  and  Holley,  1982;  Dansereau, 
McDonald,  Collins,  Garland,  Holley,  Diekhoff  and  Evans,  1979;  Holley  and 
Dansereau,  1984)  is  a  text  mapping  strategy  developed  as  part  of  an  overall  study 
strategy.  To  create  a  network  one  first  identifies  the  key  concepts  in  a  subject  area, 
and  then  displays  the  interrelationships  between  these  concepts  using  labelled 
lines  (see  example  in  Figure  7).  Currently,  networking  employs  six  linkages 
which  describe  the  relationships  between  concepts:  "part  of,"  "type  of,"  "leads  to," 
"analogy,"  "characteristic,"  and  "evidence"  (Holley  and  Dansereau,  1984).  The 
first  letter  of  each  of  these  links  is  used  to  depict  the  type  of  relationship  between 
concepts. 


Insert  Figure  7  about  here 


Schematizing  (Mirande,  1984;  Camstra  8c  Van  Bruggen,  1984)  is  another 
technique  that  uses  concepts  representing  main  ideas  linked  to  other  concepts  via 
symbolic  label s.^Jlather  than  specifying  precise  concept  relationships  the  labels 
used  in  schematizations  are:  dynamic  relationship,  static  relationship, 
similarity,  interaction,  and  the  negation  of  these  four  types  of  relationships.  These 
relationships  are  symbolized  by  different  types  of  lines  linking  concepts.  Plain 
lines,  lines  with  arrows  at  one  or  at  both  ends  and  double  lines  are  used  between 
concepts  on  schematizations.  Schematizations  include  "specifications  lists," 
which  are  lists  and  definitions  of  each  of  the  concepts  used  in  the  schematization 
graphic  (see  Figure  8  for  example). 


Insert  Figure  8  about  here 


Anderson  (1979)  and  Armbruster  and  Anderson  (1984)  developed  text  map$  as  a 
technique  to  allow  students  to  link  ideas  together  and  show  their  relationships. 
Maps  consist  of  a  general  organizing  structure,  referred  to  as  a  frame.  Within  this 
frame  learners  map  the  relationships  between  ideas  using  symbols  to  represent 
seven  different,  types  of  relationships  that  can  exist  between  two  ideas,  or  two  special 
relationships  which  define  or  highlight  the  importance  of  a  concept  (see  Figure  9  for 
example). 


Insert  Figure  9  about  here 


Selecting  Appropriate  Graphic  Techniques:  A  Model 
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Selection  of  a  graphic  techniques  to  enhance  learning  should  be  based  upon  the 
desired  performance  outcomes.  When  examining  the  different  effects  and 
outcomes  of  these  techniques  it  is  helpful  to  consider  the  cognitive  processes  and 
strategies  that  are  elicited  when  the  representations  are  generated.  We  propose  thet 
different  cognitive  processes  are  induced  by  the  cognitive  strategies  undertaken 
when  creating  the  graphic.  Cognitive  processes  are  the  manner  of  thinking  and 
associating  ideas,  including  analysis,  elaboration,  and  integration  of 
information.  Cognitive  strategies  are  the  specific  mental  manipulations  of 
information  that  are  undertaken  as  part  of  the  cognitive  process.  These  in  turn 
produce  different  effects  and  result  in  learning  at  different  levels  of  cognitive 
performance.  This  model  for  selecting  appropriate  graphic  techniques  interrelates 
the  cognitive  processes  of  the  various  spatial  representations  with  cognitive  effects 
and  performances,  and  is  presented  in  Table  1. 


Insert  Table  1  about  here 


Cognitive  Processes  and  Strategies 

As  noted  previously,  we  propose  that  the  most  important  characteristic  of 
graphic  learning  techniques  is  whether  they  represent  hierarchical  or 
heterarchical  relationships.  Further,  we  propose  that  this  difference  is  important 
because  the  development  of  hierarchical  and  heterarchical  graphic  techniques 
elicit  different  cognitive  processes. 

Analysis  and  elaboration  are  the  types  of  cognitive  processes  that  are  used  when 
developing  hierarchical  representations,  such  as  semantic  maps,  semantic 
features  analysis,  structured  overviews,  and  spider  maps.  Analysis  is  the 
breaking  down  of  concepts  into  component  parts,  and  the  organization  and 
categorization  of  those  parts.  This  type  of  cognitive  process  is  used  in  making  all  of 
the  representations,  but  is  particularly  evident  in  the  representations  which  have 
hierarchical  formats.  For  example,  note  that  the  semantic  map  shown  in  figure  3 
breaks  down  the  large  concept  of  "music"  into  important  subordinate  concepts,  and 
further  analyzes  by  identifying  organizing  those  concepts  into  groups,  and 
categorizing  them. 

The  second  type  of  cognitive  process  used  in  generating  graphic  techniques  is 
elaboration.  Elaboration  is  relating  new  knowledge  to  prior  knowledge.  There  are 
three  elaboration  strategies  that  can  be  used  when  generating  these  graphic 
techniques:  paraphrasing  material,  using  imagery  to  picture  new  material  in 
familiar  contexts,  and  generating  examples  of  new  ideas  from  prior  knowledge. 

Heterarchical  representations  include  concept  maps,  pattern  notes,  text  maps, 
networks  and  schematizations.  These  representations  use  both  analysis  and 
elaboration  but  also  allow  for  a  third  cognitive  process:  integration.  Integration 
requires  the  learner  to  draw  relationships  between  pairs  of  ideas  from  a  new 
content  dom  ain.  Integration  can  be  accomplished  by  simply  identifying 
interrelationships  between  the  concepts  displayed  in  the  graphic.  Alternatively, 
generation  of  analogies  and/or  metaphors  requires  the  learner  to  consider  the 
interrelationships  between  new  ideas  and  relate  these  new  ideas  to  prior 
knowledge. 
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Effects  and  Outcomes  of  Graphic  Techniques 

When  considering  the  effects  of  graphic  techniques  it  is  helpful  to  frame  the 
effects  in  terms  of  learning  outcomes.  One  taxonomy  of  learning  outcomes  is 
provided  by  Component  Display  Theory  (CDT)  (Merrill,  1983).  In  CDT  there  are 
three  levels  at  which  concepts,  principles  or  procedures  can  be  learned:  simple 
recall  is  referred  to  as  the  "remember"  level;  application  of  content  is  at  the  "use" 
level;  and  discovery  of  new  concepts,  principles  or  procedures  requires  that  one 
"find"  new  knowledge. 

We  propose  that  learning  tools  that  require  analysis  promote  recall  of  content. 
Breaking  "big"  concepts  into  their  component  parts  and  organizing  those  parts 
should  assist  learners  in  remembering  concepts.  Prior  research  has  indicated  that 
categorizing  ("chunking")  concepts  based  upon  similarities  promotes  recall  of 
those  concepts.  Therefore  technique  of  analysis  seems  most  helpful  in  promoting 
recall  of  content,  which  would  allow  learners  to  perform  at  the  "remember"  level  of 
learning. 

Elaboration  strategies  are  an  important  part  of  learning  because  as  new 
information  is  linked  to  prior  knowledge  it  becomes  imbedded  in  existing 
knowledge  structures,  making  it  easier  to  recall  and  use  the  information.  Mayer 
(1980,  1988)  found  that  elaborations  enhance  learners'  ability  to  transfer  learning 
to  new  situations.  In  particular,  it  seems  that  elaborations  such  as  paraphrasing 
and  exemplifying  are  helpful  in  near  transfer  tasks  in  which  the  learning  task  is 
similar  to  those  tasks  presented  in  learning  materials.  In  CDT  terminology, 
ability  to  transfer  learning  to  new  problems  in  familar  contexts  requires  learners 
to  "use"  that  information.  Since  elaboration  techniques  are  also  helpful  in 
recalling  information,  this  model  for  selecting  graphic  techniques  piedicts  that 
elaboration  strategies  would  enhance  learning  at  both  the  "remember"  and  "use" 
levels. 

Since  hetcrarchical  representations  incorporate  both  analysis  and  elaboration, 
recall  and  transfer  of  learning  are  facilitated  when  these  techniques  are  used.  In 
addition,  integration  of  information  helps  to  improve  a  learner's  ability  to  draw 
inferences  based  upon  that  content  (Mayer  1980, 1988).  Inference  is  an  important 
component  of  problem  solving,  and  is  requisite  for  learning  at  the  "find"  level. 

To  use  this  model  for  selecting  between  the  various  graphic  techniques  first 
consider  the  type  of  instructional  outcome  desired.  If  the  objective  is  simply  to 
recall  the  content  at  a  later  date  appropriate  graphic  techniques  would  be 
hierarchical.  If  one  must  recall  the  content  and  be  able  to  apply  the  knowledge  to 
new  situations  transfer  of  learning  is  required.  In  this  case  hierarchical 
techniques  are  appropriate,  but  learners  should  be  encouraged  to  elaborate  when 
developing  the  graphic.  Finally,  if  learners  are  expected  to  creatively  solve 
problems  after  completing  instruction,  appropriate  learning  techniques  should 
require  integration  of  concepts.  Therefore,  heterarchical  graphic  techniques  would 
be  most  appropriate. 


Summary 

Structural  knowledge  is  an  important  component  of  learning  in  any  content 
area.  Graphic  techniques  illustrate  the  relationships  between  concepts,  thereby 
conveying  structural  knowledge.  These  techniques  can  be  used  as  learning  tools  to 
assist  learners  in  acquiring  structural  knowledge  and  enhancing  learning 
outcomes. 

This  paper  presented  a  model  which  proposes  different  effects  of  these  graphing 
strategies  based  upon  the  cognitive  processes  required  to  construct  the  graphic. 
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Hierarchically  organized  graphics  require  analysis  and  elaboration  of  content, 
and  seem  to  enhance  recall  and  transfer  of  learning.  Heterarchical  graphics 
require  integration  of  content,  which  facilitates  inference  and  therefore  problem 
solving.  While  it  is  possible  that  other  characteristics  of  the  graphic  techniques 
may  affect  learning  outcomes,  it  is  our  contention  that  the  primary  differences  will 
result  from  the  types  of  cognitive  processes  induced  when  generating  the  graphics. 
However,  research  is  needed  to  validate  this  model  of  strategy  selection.  In 
addition,  research  into  the  differential  effects  of  learning  strategies  for  learners 
with  different  aptitudes  will  assist  instructional  designers  in  prescribing 
appropriate  learning  tasks  to  optimize  performance. 
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FACTOR  ANALYSES  OP  THE  INSTRUCTIONAL  MOTIVATION  NEEDS 

OF  ADULT  LEARNERS 


Previous  research  suggests  that  motivation  is  related  to  interest 
and  effort  and  that  for  some  learners  satisfaction  is  closely 
related  to  ones  overall  interest  in  an  instructional  environment 
In  order  to  learn  more  about  the  motivational  factors  of  adult 
learners,  data  were  collected  regarding  the  instructional 
motivation  perceptions  of  adults  in  a  variety  of  learning 
environments.  Factor  analyses  of  these  data  provide  for  some 
support  for  the  categories  of  the  ARCS  Model  for  Design  of 
Motivating  Instruction.  Adults  in  university  courses  and 
workshops  were  found  to  have  much  different  instructional 
needs  than  adults  in  community  workshop  classes.  The 
responses  of  the  subjects  in  community  classes  provided  less 
support  for  the  categories  of  Keller's  work.  The  results, 
analyses,  and  implications  are  discussed. 


A  number  of  researchers  (Aslanian  &  Brickel,  1980;  Cross,  1981;  Houle,  1961; 
Knowles,  1980;  Zemke  &  Zemke,  1981)  have  in  the  past  proposed  that  adult  learners 
have  very  specific  motivational  needs  in  instructional  settings.   These  needs  may 
be  the  result  of  life  experiences  (Knowles,  1980),  transitions  in  life  (Cross,  1981),  or 
learned  attitudes  over  time  as  a  major  factor  in  adult  learning  (Wlodkowski,  1985). 
As  a  result,  these  authors  recommend  that  instruction  for  adult  learners  be 
designed  differently  than  other  types  of  instruction. 

More  recently  Galbraith  (1990  &  1991)  has  built  on  this  theory  base, 
hypothesizing  that  facilitators  and  adult  learners  ought  to  be  engaged  in  active, 
challenging,  collaborative,  critically  reflective,  and  transforming  educational 
encounters  into  a  transactional  process.   The  guiding  principals  focus  on  (a)  an 
appropriate  philosophical  orientation;  (b)  the  diversity  of  adult  learners;  (c)  a 
conducive  psychosocial  climate;  (d)  the  need  for  challenging  interactions;  (e)  the 
promoting  critical  reflection;  and  (f)  the  encouragement  of  independence. 

While  the  literature  is  rich  with  material  about  the  needs  of  adult  learners, 
there  appears  to  be  almost  no  evidence  of  actual  research  findings  aimed  at  the 
identification  of  the  instructional  motivation  needs  of  adults.    Recently,  Bohlin, 
and  Viechnicki  (1991)  found  that  for  a  group  of  college  students,  motivation  had  two 
distinct  components  interest  and  effort.  The  analyses  of  their  data  also  suggested 
some  preliminary  support  for  the  categories  of  the  ARCS  Model  for  Design  of 
Motivating  Instruction  -  attention,  relevance,  confidence,  and  satisfaction.  The 
purpose  of  this  study  therefore  was  to  investigate  the  primary  factors  of  motivation 
for  adults  of  various  types  in  instructional  environments. 

MOTIVATIONAL  INSTRUCTION  .'iL  DESIGN 

Instructional  motivation  attracts  learners  toward  the  instruction  and  increases 
their  effort  in  relation  to  the  subject  (Keller,  1983).  This  means  that  instructional 


motivation  has  two  components,  it  is  interesting  and  effort  generating. 
Motivational  instruction,  therefore,  has  appeal  or  interest  for  the  learner  and 
stimulates  learner  effort.   This  dual  characteristic  of  instructional  motivation  has 
been  supported  in  previous  studies  (Bohlin,  Milheim,  &  Viechnicki,  1990a  &  b). 

Keller  (1987),  Keller  and  Suzuki  (1988),  and  Keller  and  Kopp  (1987)  also  identify 
four  categories  of  motivation  in  learning  situations:   attention,  relevance, 
confidence,  and  satisfaction  (ARCS).    To  facilitate  continuing  motivation, 
strategies  in  these  four  categories  should  be  addressed.  The  ARCS  model  contains 
specific  methods  or  strategies,  that  are  aimed  at  producing  motivational  outcomes, 
when  learners  are  lacking  sufficient  conditions,  such  as  interest  or  motives.  The 
initials  of  these  four  categories  (attention,  relevance,  confidence,  and  satisfaction) 
give  Keller's  model  the  acronym  ARCS. 

The  initial   requirement  for  motivating  instruction  is  to  gain  and  maintain  the 
attention  of  the  learner.  This  can  be  achieved  through  procedures  that  take 
advantage  of  the  curiosity,  interest,  or  arousal  of  the  students  by  using  humor, 
variety,  enthusiasm,  etc.  Second,  the  instruction  must  have  perceived  relevance  to 
the  immediate  or  long-range  personal  needs  of  the  learner.  These  personal  needs 
can  be  met  by  matching  the  instruction  to  learners'  goals,  making  the  benefits 
clear,  keeping  the  challenge  level  appropriate,  etc.  Next  the  instruction  must 
provide  for  the  confidence  of  the  learner.   The  instruction  must  promote  the 
learner's  expectancy  for  success  or  failure,  which  influences  the  actual  effort  and 
performance,  and  can  be  increased  by  strategies  such  as  clearly  indicating  the 
requirements  for  success,  providing  a  low  risk  environment,  and  giving  accurate 
attributional  feedback.  Lastly,  the  instruction  should  provide  individual  satisfaction 
in  order  to  facilitate  continuing  motivation.   Learners  must  perceive  ths  rewards 
gained  as  fair,  equitable,  and  consistent  while  meeting  their  expectations.  Learner 
satisfaction  can  be  promoted  by  providing  appropriate  recognition  for  success, 
giving  informative  and  corrective  feedback,  etc. 

These  strategies  and  categories  were  the  basis  for  the  development  of  two  needs 
assessment  instruments.    The  instruments,  emphasizing  interest  and  effort ,  were 
used  in  this  study.  The  research  questions  are  (a)  What  are  the  primary 
motivational  factors  for  adults  in  instructional  settings?  (b)  Are  these  factors 
different  for  adult  college  students  and  adults  from  the  general  population?  (c) 
How  do  these  students'  needs  differ  in  regards  to  interest  and  to  effort? 

METHODOLOGY 

Instruments 

The  instruments  used  in  this  study  were  the  Course  Interest  Survey  Revised 
(CISR)  and  the  Course  Effort  Survey  Revised  (CESR).  These  instruments  were 
developed  through  a  series  of  revisions  (Bohlin,   Milheim,   &  Viechnicki,  1990a; 
Bohlin,  Milheim,   &  Viechnicki,  1990b;  and  Viechnicki,  Bohlin,  &  Milheim,  1990). 
The  original  instrument  was  developed  by  Keller  and  Subhiyah  (1987)  to  evaluate 
the  percieved  degree  of  motivational  effects  of  instructional  materials.  After 
rewording  the  items  and  deleting  those  items  not  consistent  with  evaluation  of 
classroom  instruction,  several  items  were  added  which  were  identified  in  the 
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literature  as  specifically  important  to  the  instructional  motivation  of  older  learners. 

The  purpose  of  these  instruments  is  to  identify  the  instructional  motivation 
needs  of  learners.  Each  instrument  is  composed  of  the  same  42  items,  which  are  a 
selection  of  strategies  which  have  been  identified  in  the  literature  as  having 
motivational  effects  on  learners  of  various  ages  (See  Table  1).   The  instruments 
only  differ  in  asking  the  subjects  to  rate  the  importance  of  each  (of  the  42  strategies) 
on  their  own  interest  and  effort ,  respectively. 

The  instruments  might  be  used  to  determine  the  motivational  needs  of  specific 
individuals  or  of  specific  classes.  They  also  can  be  used  to  measure  the  needs  of  a 
representative  group  of  a  population  with  the  intent  of  obtaining  generalizable 
prescriptions  for  that  population. 


The  subjects  in  Phase  1  of  this  study  were  a  mixture  of  183  graduate  and 
undergraduate  students  enrolled  in  credit  and  non-credit  classes  at  four  medium- 
sized  state  universities  across  the  U.S.    These  classes  were  offered  through  Colleges 
of  Education,  and  represented  a  variety  by  content  and  methods  of  delivery.  The 
average  age  of  the  subjects  in  this  sample  was  38  years.  Fifty-seven  percent  of  the 
subjects  were  female.  These  students  responded  to  the  CISR  and  the  CESR. 

The  subjects  in  Phase  2  of  this  study  were  147  students  enrolled  in  community 
adult  education  classes  in  a  suburb  of  a  large  midwestern  city.    These  classes 
represented  a  wide  variety  by  content  and  methods  of  delivery.  The  average  age  of 
the  subjects  in  this  sample  was  46  years.  Fifty-one  percent  of  the  subjects  were 
female.  These  subjects  were  only  given  the  CISR. 

While  return  rates  were  approximately  96%  for  phase  1  and  86%  for  phase  2,  an 
additional  6%  and  3%,  respectively,  of  the  returned  response  sheets  were  discarded 
due  to  incomplete  or  out  of  range  responses. 


The  instruments  were  administered  during  regular  class  time  and  collected. 
Responses  were  made  on  scan  sheets.  Sheets  were  either  scanned  or  hand  entered  to 
make  the  responses  available  for  analysis.   Data  were  analyzed  using  Statview 
512+  on  a  Macintosh  LC  computer.   Analyses  included  means  and  standard 
deviations  for  each  of  the  items.  Comparisons  of  the  means  identified  those  items 
that  were  rated  as  most  important  to  the  subjects'  interest  and  effort  in  instructional 
settings. 

Additional  analyses  included  orthogonal  transformation  solution-varimax 
factor  analysis.   These  analyses  were  performed  in  order  gain  some  insight  into 
the  nature  of  the  response  patterns  of  the  subjects  and  to  examine  the  degree  of 
support  for  the  ARCS  model  as  identifiable  categories. 


Phase  1  -  College  students 

Analyses  of  the  responses  to  the  items  showed  that  the  strategies  were  ranked 
highly  by  the  subjects.  Several  were  rated  very  highly  by  subjects  on  both  sets  of 


Subjects 


Analyses 


RESULTS 
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instruments.  Others  were  rated  relatively  high  on  only  one  of  the  instruments.  A 
listing  of  those  highly  rated  items  (the  letter  and  number  identify  the  subscale  and 
subscale  item  number)  for  each  of  the  instruments  follows. 

Strategies  reported  as  most  important  to  the  subjects*  interest  (in  order  of  reported 
importance): 

C6    Requirements  for  success  are  made  clear  to  me 

Rl     Information  I  learn  will  be  useful  to  me 

R3     Benefit  from  the  knowledge  acquired  in  the  class 

S9     Feel  satisfied  with  what  I  learn 

Al    Makes  me  feel  enthusiastic  about  subject 

C9    Get  enough  timely  feedback  to  know  how  well  I  am  doing 

S8     Amount  of  work  I  have  to  do  is  appropriate 

CIO  Instructor  models  and  demonstrates  proper  skills  during  instruction 

R12  Personally  benefit  from  what  I  learn  in  the  class 
Strategies  reported  as  most  important  to  the  subjects1  effort  (in  order  of  reported 
importance): 

C6    Requirements  for  success  are  made  clear  to  me 

Rl    Information  I  learn  will  be  useful  to  me 

R3     Benefit  from  the  knowledge  acquired  in  the  class 

S2     Can  set  and  achieve  high  standards  of  excellence 

Al    Makes  me  feel  enthusiastic  about  subject 

S8     Amount  of  work  I  have  to  do  is  appropriate 

C9    Get  enough  timely  feedback  to  know  how  well  I  am  doing 

C4    Whether  or  not  I  succeed  is  up  to  me 
The  three  highest  ratings  by  the  subjects  in  this  phase  were  the  same  (in  order) 
for  both  instruments:  (a)  clearly  stated  requirements  for  success,  (b)  learning  of 
personally  useful  information,  and  (c)  knowledge  acquired  is  beneficial.  Three 
other  items  were  ranked  highly  on  both  instruments:  (a)  makes  me  feel 
enthusiastic  about  subject;  (b)  get  enough  timely  feedback  to  know  how  well  I  am 
doing;  and  (c)  amount  of  work  I  have  to  do  is  appropriate 

Results  of  the  factor  analyses  showed  that  for  the  CESR  the  first  four  factors 
were  principally  the  four  categories  of  strategies  proposed  by  the  ARCS  model- 
Confidence,  Relevance,  Attention,  and  Satisfaction,  respectively  (See  Table  5).  For 
the  CISR,  results  were  less  definite.  Further  analysis,  showed  that  removal  of  the 
Satisfaction  category  items  from  the  data  before  performing  the  factor  analysis 
resulted  in  a  very  clean  separation  of  items  from  the  other  categories  into  the  first 
three  factors  of  Confidence,  Relevance,  and  Attention,  respectively  (See  Table  6). 

Phase  2  -  Community  education  students 

Analysis  of  the  items  showed  that  most  items  were  rated  very  high  on  the  CISR  by 
these  subjects  (the  CESR  was  not  administered  to  this  population).  The  responses 
showed  a  pattern  of  higher  ratings  than  the  college  students  on  nearly  every  item.  A 
ceiling  effect  may  have  effected  these  data,  as  many  items  had  means  within  one 
standard  deviation  of  the  highest  possible  rating.  A  listing  of  the  items  most  highly 
rated  (the  letter  and  number  identify  the  subscale  and  subscale  item  number)  follows 

181  ;( 


Strategies  reported  as  most  important  to  the  subjects'  intercut  (in  order  of  reported 
importance): 

Si     Gives  me  a  lot  of  satisfaction 

Rll  Content  relates  to  my  expectations  and  goals 

CIO  Instructor  models  and  demonstrates  proper  skills  during  instruction 
C5     Creates  a  relaxed  classroom  atmosphere 
C4     Whether  or  not  I  succeed  is  up  to  me 
C3     Builds  my  self-esteem 

R12  Personally  benefit  from  what  I  learn  in  the  class 

C2     Makes  me  feel  I  have  the  ability  to  succeed 

Rl     Information  I  learn  will  be  useful  to  me 
Results  of  the  factor  analyses  showed  that  for  the  CISR,  the  factors  were  mixed. 
Further  analysis,  with  these  responses  showed  that  removal  of  the  Satisfaction 
category  items  from  the  data  before  performing  the  factor  analysis  resulted  in  a 
better  placement  of  items,  but  the  factors  were  not  as  "pure"  as  expected  (See  Table 


All  42  of  the  strategies  in  the  instruments  received  mean  responses  by  all  of  the 
subjects  in  the  range  of  important  to  necessary  to  their  interest  or  effort  in 
instructional  settings.  While  both  scales  shared  some  of  their  highest  rated  items 
for  college  students,  six  of  the  top  nine,  there  were  definite  differences  in  the  results 
of  the  factor  analyses  between  the  two  instruments  and  between  the  two  groups. 

The  results  of  the  first  factor  analysis  (of  the  effort  responses  of  learners  in 
college  classes)  gave  some  support  to  the  categories  of  the  ARCS  model  with  the  each 
of  the  first  four  factors  entirely  or  predominately  composed  of  items  from  one 
category  each  -  Confidence,  Relevance,  Attention,  and  Satisfaction  respectively 
(See  Table  6).  This  suggests  that  the  theoretical  nature  of  the  categories  in  the  ARCS 
model  are  consistent  with  the  nature  of  the  self-reported  motivational  needs  of 
adults  in  college  courses  and  workshops.  This  also  supports  the  traditional  type  of 
definition  of  motivation  which  usually  refers  to  time  on  task  or  some  other  measure 
of  effort. 

The  results  of  the  second  factor  analysis  (of  the  interest  responses  of  learners  in 
college  classes)  gave  factors  that  were  highly  mixed.   Repeating  the  factor  analysis 
after  removing  the  Satisfaction  items  yielded  a  very  good  separation  of  category 
items  into  the  first  three  factors     Confidence,  Relevance,  and  Attention  (See  Table 
5).   This  suggests  that  when  considering  learners'  interest,  satisfaction  may  be 
intimately  interrelated  with  the  other  categories'  strategies.   Another  explanation 
might  be  that  these  learner?  find  it  difficult  to  differentiate  the  effects  of 
satisfaction-oriented  strategies  from  others  when  referring  to  their  own  interests. 

The  fact  that  the  top  three  responses  to  both  instruments  were  the  same  may 
suggest  that  instructional  strategies  perceived  as  the  most  important  for  interest  are 
also  important  for  effort   An  alternative  explanation  is  that  for  very  important 
strategies,  subjects  have  difficulty  differentiating  between  the  effects  of  interest  and 
effort. 
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A  comparison  of  the  highest  rated  items  on  the  CISR  for  the  two  different  groups 
show  striking  differences.   The  adults  in  the  community  education  workshops  rated 
nearly  all  of  the  items  as  much  more  important.  This  supports  that  hypothesis  these 
adults  from  the  general  population  have  much  lower  levels  of  intrinsic  motivation 
in  classes,  and  feel  a  greater  need  for  extrinsic  motivation.   In  addition,  these 
learners  report  a  greater  need  for  confidence-building  strategies  -  five  of  the  top 
nine  items  were  from  the  category  of  confidence.  This  suggests  that  the  lower 
intrinsic  motivation  of  theses  learners  may  be  related  to  lower  level  of  confidence 
in  their  ability  to  succeed  in  a  formal  learning  environment. 

The  omission  of  attention-related  strategies  from  the  highest  rated  items  for  the 
non-college  adults,  can  be  explained  by  the  fact  that  they  are  taking  classes  that  are 
completely  optional,  ones  that  they  have  probably  selected  because  of  a  strong 
interest  in  the  topic.  They  therefore  would  not  need  strategies  intended  to  hold  their 
attention,  in  spite  of  reported  shorter  attention  spans  for  older  adults.  This  is 
further  supported  by  the  fact  that  the  second  highest  item  for  this  group  was  that  the 
"content  relates  to  my  expectations."  Their  interest  is  in  fact  very  dependent  upon 
how  closely  the  content  matches  their  expectations  when  enrolling  in  the  class.  The 
uniquely  high  ranking  of  the  confidence  item  that  referred  to  building  the  learners' 
self-esteem  was  noteworthy.  It  would  seem  that  adults  from  the  general  population, 
have  lower  self-esteem  and  therefore  have  a  greater  need  for  instructor  facilitation 
in  this  regard. 

The  three  highest-rated  items  for  the  adults  in  the  college  classes  were  not 
among  the  highest  rated  items  for  the  other  group.  The  first,  that  the  requirements 
for  success  are  clear,  is  probably  related  to  grades  and  their  importance  to  that 
group.  The  second,  that  information  will  be  useful,  and  the  third,  that  they  benefit 
from  the  knowledge  obtained  in  the  class,  reflect  on  the  usual  relevance  of  college 
courses  to  students'  occupations. 
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Table  1 

Content  of  items  in  both  the  Course  Interest  Survey  Revised  (CISR)  and  the 
Course  Effort  Survey  Revised  (CESR)  by  subscales 


Attention:  1  Makes  me  feel  enthrsiastic  about  subject 

2  Content  captures  my  attention 

3  Makes  the  subject  matter  seem  important 

4  Shows  how  the  content  relates  to  things  I  already  know 

5  Uses  humor  during  instruction 

6  Makes  me  feel  curious  about  the  subject  matter 

7  Does  unusual  or  surprising  things  that  are  interesting 

8  Uses  an  interesting  variety  of  teaching  techniques 

9  Curiosity  is  often  stimulated  by  the  questions  asked  or  the  problems  given 

Relevance:         1  Information  I  learn  will  be  useful  to  me 

2  Allows  time  for  practical  application  of  the  content 

3  Benefit  from  the  knowledge  acquired  in  the  class 

4  Actively  participate  in  the  class 

5  Positive  role  models  be  presented  to  me  in  class 

6  Is  flexible  to  meet  my  needs  in  content  and  assignments 

7  Personal  benefits  of  the  course  are  made  clear  to  me 

8  Challenge  level  is  about  right 

9  Have  some  input  or  choice  in  content  and  assignments 

10  Get  a  chance  to  work  with  other  people  in  the  class 

1 1  Content  relates  to  my  expectations  and  goals 

12  Personally  benefit  from  what  I  learn  in  the  class 

Confidence:        1  Helps  me  feeT  confident  that  I  can  do  well 

2  Makes  me  feel  I  have  the  ability  to  succeed 

3  Builds  my  self-esteem 

4  Whether  or  not  I  succeed  is  up  to  me  . 

5  Creates  a  relaxed  classroom  atmosphere 

6  Requirements  for  success  are  made  clear  to  me 

7  Frequent  opportunities  to  succeed 

8  Helps  me  to  believe  I  can  succeed  if  I  try  hard  enough 

9  Get  enough  timely  feedback  to  know  how  well  I  am  doing 

10  Instructor  models  and  demon Rtrates  proper  skills  during  instruction 

1 1  Non-threatening 

12  Designed  so  that  everyone  can  succeed 

Satisfaction:       1  Gives  me  a  lot  of  satisfaction 

2  Can  set  and  achieve  high  standards  of  excellence 

3  Fair  recognition  compared  to  other  students 

4  Instructor's  evaluations  of  my  work  match  how  well  I  think  I  have  done 

5  Helps  me  to  accomplish  my  own  personal  goals 

6  Feel  satisfied  with  how  the  class  is  run 

7  Get  enough  recognition  for  my  work  through  feedback 

8  Amount  of  work  I  have  to  do  is  appropriate 

9  Feel  satisfied  with  what  I  learn 
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Table  2 

Means  and  standard  deviations  of  responses  to  each  item  in  subscales  in  the  Course 
Interest  Survey  Revised  (CISR)  by  students  enrolled  in  college  courses  or  workshops 


Sequence 


Number 

Attention 

Relevance 

Confidence 

Satisfaction 

1 

1.95  (.87) 

1,80  (.80) 

2.29  (.99) 

2.o7(.o9) 

o 

O  f\C  (  QO\ 
2.Uo(.8Z) 

O  Aft  /  flJ\ 

2.U9  (.94; 

O  OA  /  1  AO  \ 

o  oo  /  on\ 

o 

0 

Z.  1Z  (.80) 

1.91  (.78) 

O  f?£  /  1  AQ  \ 

2.7o(l.Uo) 

O  KO  /  CkH\ 

Z.oz(.97) 

A 

4 

2.61  K.&6) 

z.yzll.Uoy 

O  OA  f  1  AQ  \ 

Z.z4(1.0o) 

OOO/  oo\ 

2.2o  i.o2) 

C 

0 

O  £Q  / 1   1  f\\ 
2.DO  V 1.  IV) 

n ni/1   1  O  \ 

O  OO  /  OR  I 

O  QQ  /  1  AO\ 

6 

2.29  (.85) 

2.47(1.08) 

1.75  (.81) 

2.55  (.89) 

7 

3.05(1.00) 

2.54  (.90) 

2.05  (.82) 

2.47  (.87) 

8 

2.34  (.98) 

2.32(1.00) 

2.53(1.01) 

2.03  (.82) 

9 

2.25(.82) 

2.84(1.03) 

2.00  (.93) 

1.93  (.78) 

10 

3.12(1.03) 

2.04  (.86) 

11 

2.30(.88) 

2.30(1.13) 

12 

2.04  (.85) 

2.47(1.17) 

Key:     1  =  necessary  or  essential  Bold  =  Top  ranked  items  in  CISR 

2  =  very  important 

3  =  important 

4  =  slightly  important 

5  =  not  important  N  =  183 
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Table  ? 

Means  and  standard  deviations  of  responses  to  each  item  in  subscales  in  the  Course 
Effort  Survey  Revised  (CESR)  by  students  enrolled  in  college  courses  or  workshops 


Sequence 

Number         Attention        Relevance       Confidence  Satisfaction 


1 

1.96  (.92) 

1.60  (.84) 

2.42  (.99) 

2.38(1.01) 

2 

2.30  (.87) 

2.25  (.89) 

2.73(1.05) 

1.92  (.83) 

3 

2.36  (.85) 

1.91  (.82) 

2.94(1.13) 

2.32(1.07) 

4 

2.59(1.04) 

2.90(1.04) 

2.07  (1.10) 

2.13  (.89) 

5 

2.91(1.12) 

3.01(1.12) 

2.49(1.03) 

3.10(1.10) 

6 

2.28  (.80) 

2.52  (.97) 

1.79  (.86) 

2.51(1.02) 

7 

3.26(1.06) 

2.74(1.16) 

2.09(1.01) 

2.47(1.04) 

8 

2.71(1.06) 

2.26  (.96) 

2.58(1.14) 

2.04  (J5) 

9 

2.45  (.96) 

3.05(1.05) 

2.05  (.89) 

2.09  (.88) 

10 

3.23(1.14) 

2.29(1.05) 

11 

2.35  (.92) 

2.52(1.15) 

12 

2. 13  (.94) 

2.63(1.17) 

Key:     1  =  necessary  or  essential  Bold  =  Top  ranked  items  in  CESR 

2  =  very  important 

3  =  important 

4  =  slightly  important 

5  =  not  important  N  =  183 
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Table  4 

Means  and  standard  deviations  of  responses  to  each  item  in  subscales  in  the  Course 
Interest  Survey  Revised  (CISR)  by  students  enrolled  in  adult  community  workshops 


Sequence 
Number 

Attention 

Relevance 

Confidence 

Satisfaction 

1 

1.59  (.66) 

129  (.80) 

1.35  (.52) 

122  (.42) 

2 

1.33  (.49) 

1.62(62) 

129  (.46) 

1.31  (.51) 

3 

1.31  (.49) 

2.03  (.53) 

128  (.47) 

1.97  (.65) 

4 

1.31  (.52) 

1.78  (.78) 

126  (.47) 

2.10C44) 

5 

1.42  (.64) 

1.57  (.62) 

125  (.64) 

1.91  (.54) 

6 

2.08  (.46) 

1.46  (.59) 

1.97  (.51) 

1.95  (.55) 

7 

2.61  (.80) 

1.42  (.60) 

2. 10  (.47) 

2.08  (.57) 

8 

1.99  (.56) 

1.41  (.53) 

1.45  (.58) 

1.87  (.62) 

9 

1.37  (.60) 

2.13(.71) 

1.72(.61) 

1.35  (.49) 

10 

2.42  (.89) 

125  (.45) 

11 

123  (.42) 

1.64  (.70) 

12 

128  (.51) 

1.63  (.72) 

Key:     1  =  necessary  or  essential  Bold  =  Top  ranked  items  in  CISIl 

2  =  very  important 

3  =  important 

4  =  slightly  important 

5  =  not  important  N  =  147 
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Table  5 

Items  arranged  and  ranked  by  weightings  in  the  top  factors 

of  the  CESR  by  students  enrolled  in  college  courses  or  workshops 


Factor  1  (Confidence) 

C8  Helps  me  to  believe  I  can  succeed  if  I  try  hard  enough 

C3  Builds  my  self-esteem 

C2  Makes  me  feel  I  have  the  ability  to  succeed 

57  Get  enough  recognition  for  my  work  through  feedback. 
CI  Helps  me  feel  confident  that  I  can  do  well 

55  Helps  me  to  accomplish  my  own  personal  goals 

C9  Get  enough  timely  feedback  to  know  how  well  I  am  doing 

Ci2  Designed  so  that  everyone  can  succeed 

Factor  2  (Relevance) 

R12  Personally  benefit  from  what  I  learn  in  the  class 

R3  Benefit  from  the  knowledge  acquired  in  the  class 

S9  Feel  satisfied  with  what  I  learn 
Rl  -  Information  I  learn  will  be  useful  to  me 

R8  Challenge  level  is  about  right 

R2  Allows  time  for  practical  application  of  the  content 

R7  Personal  benefits  of  the  course  are  made  clear  to  me 

Factor  3  (Attention) 

CIO  Instructor  models  and  demonstrates  proper  skills 

A8  Challenge  level  is  about  right 

A3  Makes  the  subject  matter  seem  important 

A5  Uses  humor  during  instruction 

R5  Positive  role  models  be  presented  to  me  in  class 

A9  Curiosity  is  often  stimulated 

Factor  4  (Satisfaction) 

53  Recognition  I  receive  is  fair 

58  Amount  of  work  I  have  to  do  is  appropriate 

S7  Get  enough  recognition  for  my  work  through  feedback 

54  Instructor's  evaluations  of  my  work  match  how  well  I  think  I  have  done 

56  Feel  satisfied  with  how  the  class  is  run 

Factor  5 

R9  Have  some  input  or  choice  in  content  and  assignments 

RIO  Get  a  chance  to  work  with  other  people  in  the  class 

S6  Feel  satisfied  with  how  the  class  is  run 
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Table  6 

Items  (with  satisfaction  items  removed)  arranged  and  ranked  by  weightings  in 
the  top  factors  of  the  CISR  by  students  enrolled  in  college  courses  or  workshops 


Factor  1  (Confidence) 
C3     Builds  my  self-esteem 

C8    Helps  me  to  believe  I  can  succeed  if  I  try  hard  enough 
CI     Helps  me  feel  confident  that  I  can  do  well 
C2    Makes  me  feel  I  have  the  ability  to  succeed 

Factor  2  (Relevance) 
R3     Benefit  from  the  knowledge  acquired  in  the  class 
Rl    Information  I  learn  will  be  useful  to  me 
R12  Personally  benefit  from  what  I  learn  in  the  class 
R8    Challenge  level  is  about  right 

Factor  3  (Attention) 
A3     Makes  the  subject  matter  seem  important 
A2    Content  captures  my  attention 
Al     Makes  me  feel  enthusiastic  about  subject 
C2    Makes  me  feel  I  have  the  ability  to  succeed 

Factor  4  (Success  &  Relevance) 
C6    Requirements  for  success  are  made  clear  to  me 
C7    Frequent  opportunities  to  succeed 
R9    Have  some  input  or  choice  in  content  and  assignments 
R7    Personal  benefits  of  the  course  are  made  clear  to  me 

Factor  5  (Fun) 
A5     Uses  humor  during  instruction 

A7  Does  unusual  or  surprising  things  that  are  interesting 
C5    Creates  a  relaxed  classroom  atmosphere 

Factor  6  (Curiosity) 
A6    Makes  me  feel  curious  about  the  subject  matter 
A9    Curiosity  is  often  stimulated 

Factor  7  (Sunporfl 
C12  Designed  so  that  everyone  can  succeed 
C 1 1  Non-threatening 

CIO  Instructor  models  and  demonstrates  proper  skills 

Factor  8  (Active) 
RIO  Get  a  chance  to  work  with  other  people 
R4    Actively  participate  in  the  clas3 
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Table  7 

Items  (with  satisfaction  items  removed)  arranged  and  ranked  by  weightings  in 
the  top  factors  of  the  CISR  by  students  enrolled  in  adult  community  workshops 


Factor  1  (Confidence) 

C4  Whether  or  not  I  succeed  is  up  to  me 

C2  Makes  me  feel  I  have  the  ability  to  succeed 

CI  Helps  me  feel  confident  that  I  can  do  well 

A4  Shows  how  the  content  relates  to  things  I  already  know 

C3  Builds  my  self-esteem 

Factor  2  (Relevance  plus) 

R7  Personal  benefits  of  the  course  are  made  clear  to  me 

A9  Curiosity  is  often  stimulated 

C8  Helps  me  to  believe  I  can  succeed  if  I  try  hard  enough 

R8  Challenge  level  is  about  right 

Factor  3  (Relaxed) 

Cll  Non-threatening 

C12  Designed  so  that  everyone  can  succeed 

Factor  4 

CIO  Instructor  models  and  demonstrates  proper  skills 

Rll  Content  relates  to  my  expectations  and  goals 

A8  Uses  an  interesting  variety  of  teaching  techniques 

Factor  5  (Success) 

C6  Requirements  for  success  are  made  clear  to  me 

C7  Frequent  opportunities  to  succeed 

RIO  Get  a  chance  to  work  with  other  people 

A7  Does  unusual  or  surprising  things  that  are  interesting 

Factor  7 

R2  Allows  time  for  practical  application  of  the  content 

A6  Makes  me  feel  curious  about  the  subject  matter 
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The  Effects  of  Count*  Structure  on  Students*  Computer  Attitudes 


This  study  investigated  the  effects  of  course  structure  variables 
on  the  computer  anxiety,  confidence,  and  attitudes  of  college 
students.  These  course  structure  variables  included  the  number 
of  weeks  the  course  met  (course  length)  and  number  of 
meetings  per  week  (course  frequency).  The  effects  of 
differences  in  instructors'  use  of  anxiety  reducing  strategies  as 
reported  by  the  students  were  controlled  by  statistical  analyses. 
Differences  in  course  structure  only  had  significant  effects  on 
students'  computer  anxiety.  Students  in  courses  that  met  more 
times  per  week  and  for  a  longer  time  period  had  their 
computer  anxiety  reduced  to  a  greater  extent.  Computer 
confidence  and  attitudes  were  not  differentially  affected. 

Anxiety  toward  computers  can  be  a  tremendous  problem  for  individuals, 
especially  teachers,  in  a  society  that  is  rapidly  becoming  more  technological. 
Sanders  and  Stone  (1986)  identify  societal  and  occupational  consequences  of 
computer  deficiencies  in  our  country.  These  results  of  computer  avoidance  have  an 
adverse  effect  on  our  society  as  it  becomes  more  dependent  upon  computers  and 
other  technological  innovations.   They  emphasize  the  importance  of  improving 
computer  skills  and  attitudes  for  all  learners.    Raub  (1981)  surveyed  college 
students  regarding  their  attitudes  toward  computers  and  reported  that  fear  or 
anxiety  about  computers  led  to  negative  attitudes  toward  their  use.  Badagliacco 
(1990)  and  Howard,  Murphy,  and  Thomas  (1987)  state  that  the  issue  of  feelings  of 
anxiety  and  of  alienation  toward  computers  is  sufficiently  important  that  intensive 
study  is  needed.  We  should  learn  more  about  the  nature  of  computer  anxiety  so  that 
we  might  better  prescribe  effective  interventions  for  learners. 

In  order  to  gain  a  better  understanding  of  the  nature  of  computer  anxiety  and 
confidence  it  is  important  to  investigate  their  interaction  with  instruction  (Bohlin, 
1990).  This  study  was  designed  to  investigate  the  effects  of  different  course 
structures  on  the  attitudes  of  students  by  investigating  student  perceptions  of  the 
computer  instruction  and  changes  in  their  computer  anxiety  and  confidence  scores. 

COMPUTER  ATTITUDES 

Computer  anxiety  can  be  viewed  as  a  form  of  state  anxiety  associated  with 
computer  use  (Cambre  &  Cook,  1985;  Howard,  1984).  Computer  anxiety  has  been 
defined  as  "the  fear  or  apprehension  felt  by  individuals  when  they  used  computers, 
or  when  they  considered  the  possibility  of  computer  utilization"  (Simonson,  Mauer, 
Tarardi,  and  Whitaker,  1987,  p.  238). 

Studies  (Hunt  &  Bohlin,  in  press;  and  Loyd,  Loyd,  AGrestard,  1987)  have 
found  a  strong  relationship  between  computer  confidence  and  computer  anxiety. 
This  relationship  is  believed  to  be  based  in  the  cognitive  link  between  expectancy 
for  success  with  computers  and  diminished  fear  of  interacting  with  them.  A 
learner  who  feels  anxious  is  less  likely  to  be  confident  of  performance. 
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Conversely,  a  computer-confident  i earner  is  less  likely  to  feel  anxiety  about  using 
or  learning  on  a  computer.  Several  studies  have  reported  significant  correlates  to 
computer  anxiety.   However  none  has  looked  at  changes  in  computer  anxiety  and 
confidence  as  related  to  specific  instructional  structure.   Loyd  AOressard  (1987) 
have  found  a  relationship  between  computer  confidence  and  anxiety  and  usefulness 
and  liking  of  computers.  Because  one  goal  of  a  computer  class  in  Teacher 
Education  is  to  facilitate  the  later  integration  of  computers  into  the  teachers' 
curriculum,  this  study  also  investigated  the  impact  on  all  four  variables. 

Because  anxiety  interferes  with  encoding  and  retrieval  of  learned  information, 
it  ie  expected  that  increasing  the  frequency  of  meetings  in  a  computer  class  would 
have  a  greater  impact  on  the  learners*  computer  confidence,  which  in  turn  would 
have  a  greater  influence  on  their  anxiety.  Also,  it  is  expected  that  increasing  the 
length  of  a  computer  class  would  have  a  greater  impact  on  the  learners1  computer 
anxiety  and  confidence.   This  Btudy  examined  the  effects  of  both. 

METHODOLOGY 

The  study  was  conducted  during  the  Spring,  Summer,  and  Pall  1991  semesters, 
using  over  15  different  sections  of  a  course  required  for  state  teaching  credentials. 
The  subjects,  ranging  in  age  from  20  to  57  years  of  age  (mean  ■  31.9),  were  381 
undergraduate,  post-baccalaureate,  and  graduate  students  at  two  campuses  of  a  very 
large  western  state  university.  The  sample  included  71  males,  292  females,  and  18 
"no  response"  pre-  and  in-service  teachers.   They  were  given  the  instruments 
during  their  first  or  second  lecture  meeting  of  the  term  and  again  during  the  last 
week  of  classes. 

The  following  tests  were  selected  as  measures  of  the  variables  investigated  in 
this  study.  Computer  anxiety,  computer  confidence,  usefulness  of  computers,  and 
liking  of  computers  were  measured  using  the  appropriate  subscales  of  the  Computer 
Attitude  Scale  (CAS),  developed  by  Loyd  &  Gressard.  The  instrument  was 
validated  by  judges  ratings  and  an  factor  analysis  of  the  ratings  of  155  sutyects, 
and  the  subscales  of  the  instrument  have  a  coefficient  alpha  reliability  ranging 
from  0.76  to  0.89  (Loyd  and  Gressard,  1984). 

Because  the  teachers'  instructional  strategies  can  also  have  an  impact  on  the 
affect  of  the  learners,  analyses  controlled  for  the  rate  of  use  of  nine  specific 
anxiety-reducing  teacher  behaviors.   Data  for  this  variable  were  obtained  by 
subjects'  recall.  Students  were  asked  to  report  the  frequency  (5-level  Likert  scale) 
with  which  their  instructor  used  the  nine  identified  strategies.     These  anxiety- 
reducing  strategies  studied  as  a  variable  in  this  study  included  the  following: 

•  Showing  the  insides  of  a  computer  or  peripheral 

•  Modeling  computer  skills 

•  Kncouraging  students  to  learn  on  the  computer  by  trial  &  error 

•  Providing  for  hands-on  computer  experiences  in  class 

•  Having  students  work  in  pairs  or  small  groups 

•  Providing  role  models  to  show  that  computers  are  not  just  a  white  male 
domain 

•  Assuring  learners  that  it  is  alright  to  make  mistakes  on  a  computer 

•  Kxplainlng  that  learners  are  really  in  control  of  the  computer 
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Information  regarding  the  age  and  Bex  of  the  subjects  was  also  obtained. 
Sections  were  coded  according  to  the  length  of  the  course  (greater  than  or  less  than 
11  weeks)  and  to  the  number  of  sessions  per  week  (once  or  more  than  once). 

Multiple  analysis  of  variance  on  repeated  measures  were  performed  with  the 
four  dependent  variables  of  computer  anxiety,  computer  confidence,  usefulness  of 
computers,  and  liking  of  computers  scores  and  with  the  two  independent  variables 
of  course  length  (long  and  short  course)  and  the  frequency  of  meeting  (high  and  low 
frequency),  and  the  three  covariates  of  instructor  behavior  (high  and  low  frequency 
of  anxiety  reducing  behaviors),  age  (high  and  low),  and  sex  (male  and  female). 

Sufcrjecte  were  directed  to  code  all  of  their  responses  on  scanable  forms.  Their 
responses  were  electronically  scanned  and  all  analyses  were  performed  using 
SPSS-PC  on  an  IBM-compatable  AT  computer. 

RESULTS 

The  Multiple  Analyses  of  Variance  found  that  changes  in  computer  anxiety 
differed  significantly  for  frequency  of  meeting  (^  a01  =8.86,  p  <  .006),  for  course 
length  (F1381  «  3.94,  p  <  .05),  and  for  frequency  of  meeting  X  course  length 
(Fi  381  «  5.08,  p  <  .05).  Students  in  high  frequency  courses  (that  met  more  than  once 
per  week)  and  long  courses  (that  met  for  greater  than  eight  weeks)  had  a  greater 
reduction  in  their  anxiety  towards  computers. 

The  Multiple  Analyses  of  Variance  on  differences  of  changes  in  students* 
computer  confidence,  perceptions  of  usefulness  of  computers,  and  liking  of 
computers  by  course  structure  were  found  to  be  not  significant 

DISCUSSION 

These  results  suggest  that  for  these  teachers  and  pre-service  teachers  the 
frequency  of  instructional  meetings  per  week  and  the  period  of  time  over  which  the 
instruction  occurs  can  have  an  important  impact  on  the  learners4  post-course 
computer  anxiety. 

For  these  subjects,  however,  frequency  of  meeting  and  length  of  course  (within 
these  limits)  do  not  appear  to  have  a  significantly  different  effect  on  the  students' 
computer  confidence,  perceptions  of  computer  usefulness,  and  liking  of  computers. 
This  is  somewhat  surprising  in  several  ways. 

The  mechanism  for  the  effects  of  high  frequency  classes  lessening  the  anxiety 
is  typically  expected  to  be  through  improved  confidence.  In  working  with  the 
computers  regularly  with  short  time  periods  between  lessons,  the  students  do  not 
have  to  recall  procedures  over  long  time  periods  such  as  one  week.  Because  anxiety 
interfers  with  memory  (both  encoding  and  recall),  structures  in  the  course  that 
allow  for  easier  recall  or  encoding  would  improve  an  anxious  learner's 
confidence.    Greater  confidence  and  less  frustration  in  forgetting  previously 
learned  procedures  would  be  expected  to  reduce  anxiety  to  a  greater  extent.  These 
results  suggest  that  the  level  and  rate  of  frustration  during  instruction  may  be  a 
stronger  intervening  variable  for  changes  in  computer  anxiety  than  those 
investigated.  It  also  might  be  important  to  investigate  the  degree  to  which 
instructors  assist  their  students1  memory  of  important  skills. 
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In  spite  of  the  lack  of  significance  of  length  of  course  on  three  of  the  variables, 
one  would  expect  that  instructional  interventions  over  a  longer  period  of  time  would 
have  a  greater  effect  on  attitudes  and  confidence.  Affective  shifts  seem  to  require  a 
relatively  period  of  long  time  to  occur.  It  1b  possible  that  changes  in  computer 
confidence  may  be  more  closely  related  to  total  time  working  with  computers  and 
less  with  these  variables.  Because  of  the  nature  of  this  course,  all  sections  meet 
about  the  same  amount  of  total  time,  Therefore,  total  instructional  time  was  not 
investigated  as  a  variable  in  this  study. 

IMPLICATIONS 

If  a  reduction  in  computer  anxiety  is  an  important  consideration  in  the  design 
of  a  course  and/or  training  for  a  similar  type  of  student,  the  instruction  should  be 
planned  so  that  it  meets  frequently  during  the  week  and  so  that  the  formal  learning 
takes  place  over  a  period  of  more  than  eleven  weeks.  These  results  suggest  that  the 
typical  summer  school  course  or  short-term  workshop  should  not  be  used  when 
working  with  in-  and  pre-eervice  teachers,  if  they  have  computer  anxiety.  These 
results  sIbo  Buggest  that  even  quarter  computer  courses,  usually  ten  weeks  in 
length,  may  not  be  adequate  for  learners  with  relatively  high  anxiety. 

Instructional  strategies  that  reduce  computer  anxiety  do  not  significantly  affect 
confidence  and  attitudes  toward  computers.  While  there  are  strong  correlations 
between  these  variables,  they  are  in  fact  affected  differentially.  It,  therefore,  would 
seem  to  be  important  to  continue  to  investigate  at  least  all  four  of  these  variables 
when  studying  students  and  their  affective  perceptions  of  computers. 

These  results  also  suggest  that  the  degree  of  frustration  during  learning  may  be 
an  important  intervening  variable  to  changes  in  computer  anxiety.   It  is 
recommended  that  this  variable  be  included  in  future  studies  of  interventions  of 
computer  anxiety. 

It  may  be  necessary  to  examine  the  rate  of  instructors'  use  of  classroom  methods 
of  supporting  students'  memory  during  instruction,  in  order  to  more  precisely 
measure  differing  effects  of  course  structure.  These  strategies  are  different  than 
the  anxiety-reducing  strategies  investigated  in  this  study. 
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Networking  on  the  Network:  Teachers  and  Electronic  Mail 
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University  of  Kansas 

The  new  communication  technologies  of  computer-mediated  communication 
(CMC),  including  electronic  messaging  (commonly  called  E-mail),  are  now 
coming  into  the  early  stages  of  adoption  in  elementary  and  secondary  schools. 
Initially  computers  were  often  adopted  for  instructional  purposes,  and  now  their 
instructional  use  is  becoming  extensive  (Cuban,  1986).  They  are  increasingly 
finding  a  new  use:  communication  networking.  (See  McCarthy,  1989,  for  a 
discussion  of  student  networking;  West,  Inghilleri,  McSwiney,  Sayers,  &  Stroud, 

1989,  for  a  technical  report  on  an  electronic  network  for  teachers;  and  McAnge, 

1990,  for  a  directory  of  computer  networks  and  networking  projects  for  teachers.) 
As  with  many  new  electronic  technologies,  there  is  little  if  any  argument  that  their 
eventual  widespread  use  is  inevitable  (Huber,  1984),  partly  because  within 
organizations,  use  of  computers  for  one  purpose  encourages  further  computer  use 
(Salem  and  Gratz,  1989). 

Schools  and  computer  communication 

The  school  culture  into  which  CMC  is  being  introduced  is  one  that  faces 
numerous  and  profound  difficulties  of  communication.  Jackson  (1968)  found  that 
many  teachers  had  "a  desire  to  draw  more  heavily  than  they  presently  do  on  the 
services  of  other  specialists  within  the  system-  such  as  music  and  art  teachers"  (p. 
133).  Some  social  or  institutional  barrier  against  a  greater  degree  of  interchange 
and  collegiality  is  implied  by  this  desire -and  its  lack  of  fulfillment. 
Feiman-Nemser  and  Floden  (1986)  found  that  teacher  isolation  was  a  reality,  and 
teachers1  interactions  seldom  included  discussions  of  work  or  collaborations  on 
shared  problems.  Lortie  (1975)  described  schools  as  having  "separate  cells"  (p.  14) 
in  which  teachers  worked,  with  "gaps  in  interpersonal  support"  (p.  73).  Almost 
half  (45  percent)  of  Lortie's  respondents  said  they  had  no  work-related  contact  with 
other  teachers,  and  nearly  a  third  (32  percent)  had  "some"  (p.  193).  Teachers  spend 
inordinate  amounts  of  time  compartmentalized  away  from  other  adults, 
surrounded  only  by  students  (The  Holmes  Group,  1986;  Sarason,  1982).  Cusick 
(1981)  noted  that  teachers  were  isolated  in  their  classrooms  and  lacked  common 
values  regarding  student  behavior;  he  found  that  the  schools  had  no  force  secving  to 
build  consensus. 

Some  of  the  above  communication  inadequacy  would  seem  to  be  particularly 
susceptible  to  solution  by  electronic  mail.  E-mail  is  fast,  Even  relatively  slow 
E-mail  systems  that  "store  and  forward"  messages,  resulting  in  overnight 
delivery,  are  much  faster  than  hard-copy  personal  letters  and  memos.  E-mail  is 
asynchronous,  meaning  it  can  function  when  communicators  are  separated  by 
distance  and  are  not  attending  to  the  exchange  at  the  same  time  (Rogers,  1988), 
unlike  other  rapid  forms  of  electronic  media  such  as  the  telephone.  Computer 
networks  have  already  had  demonstrable  effects  on  business,  including 
encouraging  communication  between  individuals  who  are  in  dispersed  locations; 
they  have  the  general  effect  of  overcoming  the  limitations  of  time,  spatial  distance, 
and  interaction  with  the  organizational  hierarchy  with  which  communication 
media  must  deal  (Rogers,  1988).  Taylor  (1986)  theorized  that  CMC  would  encoura^i 
horizontal  contacts  (those  outside  the  normal  superior-subordinate  relationship 
within  departments),  and  would  fundamentally  challenge  and  change 
organizational  structure,  creating  a  freer,  less  hierarchical  organizational 
structure.  Communication  mechantHms  such  as  electronic  mail,  which  help 
overcome  the  logistical  difficulty  of  teacher  interpersonal  communication,  may 
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have  a  considerable  impact  on  the  overall  type  and  quantity  of  communicating 
teachers  do  with  each  ether. 

Information  potential  of  communication 

Generally,  people  who  are  homophilous  (belong  to  the  same  group,  as  in 
religion,  ethnic  background,  profession  or  other  categorical  variable)  have 
stronger  relationships  with  each  other  than  those  who  are  heterophilous  (not  alike 
on  the  categorical  variable  of  interest)  (Granovetter,  1973;  1982).  According  to  the 
theory  of  the  "strength  of  weak  ties,"  there  is  a  price  to  be  paid  for  having  strong 
ties:  They  do  not  allow  for  as  much  transmission  of  innovative  and  novel  ideas  as 
weak  ties  (those  between  individuals  who  are  less  similar). 

Granovetter  argued  that  a  person  ("Ego")  will  have  friends  (to  whom  Ego 
relates  closely)  who  are  more  likely  to  know  each  other  well  than  are  Egos 
acquaintances  (to  whom  Ego  relates  less  closely)  -  and  therein  may  lie  a  type  of 
social  inbreeding  of  ideas.  Ego's  close  friends  form  a  relatively  dense  clump,  but 
Ego's  acquaintances  probably  have  their  own  close  friends,  and  thus  their  own 
dense  clumps  of  social  ties  different  from  Ego's.  People  with  few  acquaintances, 
under  the  theory  of  the  strength  of  weak  ties,  get  less  information  and  input  from 
people  other  than  their  close  friends,  and  thus  may  have  a  more  provincial  store  of 
news  and  opinion  to  draw  from  (Granovetter,  1982). 

Put  in  more  formal  terms,  the  strength  of  weak  ties  means  there  is  a  higher 
information-exchange  potential  in  dyadic  communication  when  the 
communicators  are  heterophilous  (Rogers  &  Kincaid,  1981).  People  are  more 
likely  to  find  out  something.new  and  useful  from  someone  who  comes  from  a  bit  of 
a  different  background. 

Within  the  world  of  education,  then,  contacts  should  be  encouraged  between 
teachers  who  come  from  different  content  areas,  since  contacts  within  content 
areas  have  less  information  potential  than  those  that  are  cross-disciplinary.  For 
example  teachers  in  English  or  foreign  languages  should  derive  utility  from  ideas 
and  methods  used  in  science,  methods  they  are  unlikely  to  encounter  without 
contact  with  science  teachers.  But  teachers'  personal  communication  networks, 
according  to  the  above-cited  literature,  do  little  to  encourage  or  accomplish  much 
interchange  at  all,  and  Rohland  (1985)  found  teachers  interact  most  often  with  those 
who  are  nearby,  and  in  the  same  teaching  content  discipline  -  in  other  words,  most 
relationships  are  homophilous  in  terms  of  teaching  content  area. 

Research  questions 

Since  adoption  of  electronic  messaging  has  begun  in  schools,  the  time  was  ripe 
for  a  study  of  the  uses  to  which  early  adopters  put  E-mail.  Early  adopters  of  new 
technology  can  play  a  very  influential  role  in  the  ultimate  uses  to  which  the 
technology  is  put.  The  study  looked  for  evidence  that  electronic  mail  encouraged 
contact  between  teachers  in  different  content  areas,  indicating  the  potential  for  a 
change  in  the  personal  networking  patterns  of  teachers  that  would  ease  the 
transmission  of  novel  ideas  and  teaching  approaches. 

•  Who  were  the  early  users  of  E-mail  among  teachers  and  how  were  they  using  it 
it? 

•  How  did  the  amount  of  usage  break  down  by  teaching  content  area? 

•  How  much  messaging  went  on  within  and  between  content  areas? 

UNITE  System 

The  system  studied  was  UNITE,  the  Unified  Network  for  Informatics  in 
Teacher  Education  (UNITE)  system  at  the  Kansas  University  Instructional 
Technology  Center,  a  computer-mediated  communciation  system  involving 
hypertext.  UNITE,  at  the  time  of  this  study,  was  operating  in  six  school  districts  and 
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16  schools  in  eastern  Kansas.  In  nearly  all  cases,  there  was  one  UNITE  terminal 
in  each  school.  Most  of  them  were  located  in  the  school  library,  a  department  office, 
or  some  work  area  accessible  to  numerous  teachers. 

UNITE  is  written  in  HyperCard  and  has  two  major  components:  an  electronic 
mail  system,  and  what  are  called  dynamic  resources,  which  take  the  form  of  a 
hypertext  shared  database  of  administrative  information,  computer  courseware, 
lesson  plans,  and  research  in  a  broad  spectrum  of  academic  content  areas.  The 
system's  original  design  was  primarily  centered  on  making  the  dynamic 
resources  available  to  teachers,  and  the  electronic  mail  component  was  added  to  the 
system  to  make  communication  easier  between  users  of  the  resources  and  between 
users  and  system  developers. 

Teachers  can  access  information  in  the  dynamic  resources,  add  lesson  plans 
and  evaluations,  and  contribute  abstracts  of  pertinent  articles  and  research,  but  it 
is  the  electronic  messaging  system  that  is  of  interest  to  this  study.  Teachers  can  use 
the  E-mail  system  to  send  one-to-one  messages  (identifying  other  specific 
individuals  on  the  system),  or  one-to-many  messages.  One-to-many  message 
senders  may  identify  either  the  members  of  special  interest  groups  (SIGs)  (for 
example,  Science  Teachers  or  Message  System  Critics),  or  they  may  send  a 
message  "to  all,"  which  targets  everyone  who  has  a  UNITE  electronic  mailbox. 

It  is  important  to  note  that,  while  the  UNITE  E-mail  system  was  originally 
intended  to  play  a  subordinate  role  to  the  development  of  the  dynamic  resources,  it 
took  on  a  life  of  its  own  very  early  in  system  development,  and  many  if  not  most  of 
the  messages  posted  (including  general  one-to-many  and  individual  one-to-one 
messages)  concerned  topics  divorced  from  dynamic  resource  development.  This 
substantiated  E-mail's  ability  to  serve  a  need  and  fill  a  niche  in  educational 
communication. 

A  further  important  descriptive  point  about  the  UNITE  environment:  The 
communication  that  took  place  on  the  syotem  was  almost  entirely  ad  hoc,  since  no 
formal  structures  or  training  for  using  the  system  were  implemented.  UNITE  s 
graphical  user  interface  was  designed  for  ease  of  use  (in  stark  contrast  to  most 
mainframe  electronic  mail  systems  such  as  run  E-mail  at  most  universities); 
technical  support  and  training  for  new  users  was  minimal,  and  many  users 
simply  logged  themselves  on,  followed  directions  to  establish  their  own  password, 
and  used  the  system  on  their  own.  Some  support  was  clearly  available  from 
teachers  who  got  involved  in  the  project  early  on  and  encouraged  others  to  join  in, 
but  that  support  was  entirely  informal  and  in  many  schools  virtually  nonexistent. 
Thus,  electronic  communication  on  UNITE  was  virtually  unguided  and 
uninfluenced  by  any  particular  networking  strategy  beyond  a  desire  to  provide  nn 
environment  in  which  teachers  could  easily  get  E-mail  messages  to  each  other. 
(See  Aust,  1991,  April;  and  Aust  and  Klayder,  1990,  February,  for  discussions  and 
descriptions  of  UNITE  by  the  systems  developers.) 

Method 

A  network  analysis  was  conducted  of  one-to-one  messaging  on  the  UNITE 
E-mail  system.  A  core  of  volunteer  was  recruited  and  their  permission  obtained 
to  study  to  whom  they  sent  messages  and  from  whom  they  received  them.  The 
researcher  was  not  given  access  to  the  actual  content  of  the  messages,  but  all 
volunteers  answered  a  questionnaire  that  characterized  their  contacts  as 
professional  or  social  in  nature.  The  who-to-whom  messaging  data  were  gathered 
electronically  and  unobtrusively  from  the  hub  of  the  UNITE  syfltom  according  to 
guidelines  and  procedures  suggested  by  Danowski  (1983). 

The  amount  of  messaging  was  measured;  the  omorgont  (informal) 
communication  groups  that  formed  on  the  system  were  identified  uning  a  computer 
program  called  NEGOPY  (Richards  &  Ilicc,  1981),  which  has  boon  recommended 
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as  a  de  facto  standard  for  cluster  analysis  (Rogers  &  Kincaid,  1981).  Where 
messages  moved  in  only  one  direction,  resulting  in  an  unreciprocated 
communication  relationship,  the  program  dropped  links  to  force  reciprocation.  The 
messaging  within  and  between  teaching  content  areas  was  compared  using 
chi-squares  with  links  formed  as  dependent  variable. 

Finally,  subjects  in  the  study  filled  out  a  questionnaire  that  included  closed- 
ended  items  about  user  demographics,  computer  literacy,  and  access  to  the  system, 
along  with  open-ended  items  concerning  why  they  used  the  system  and  what  they 
got  out  of  it  These  responses  were  analyzed  somewhat  informally,  but  produced 
some  useful  information  in  light  of  the  communication  structures  identified  by  the 
network  analysis. 

Results 

Sixty-five  UNITE  system  users  agreed  to  participate  in  the  study;  42  of  them 
(65%)  used  the  E-mail  system  either  to  send  or  receive  one-to-one  messages;  32 
(49%)  were  message  senders. 

Content  areas:  The  largest  group  of  E-mail-using  teachers  was  the  science 
teachers  (15  or  36%),  with  a  substantial  contingent  of  librarians  and  educational 
communication  specialists  (10  or  24%);  there  were  also  university/adult  education 
teachers  (6  or  14%),  general  elementary  teachers  (5  or  12%),  and  6  (14%) 
miscellaneous  others. 

IvfoflRftfnnfT  sending:  Easily  the  most  avid  users  of  the  E-mail  system  were  the 
librarians.  They  corresponded  on  the  system  with  more  individuals  than  non- 
librarians  (an  average  of  4.8  links  per  librarian,  compared  to  2.5  for  all  other 
E-mail  users)  and  sent  more  messages  (an  average  of  56.6  compared  to  8.5). 

Those  who  had  a  UNITE  terminal  on  their  desk,  in  the  same  room,  or  in  a 
nearby  room  (n  =  21)  sent  significantly  more  messages  than  those  for  whom  the 
terminal  was  not  so  close  (n  =  21)  (Mann-Whitney  test,  U  =  145.0,  W  =  376.0, 
Z  =  1.9212,  p  <  .05). 

Punster  analysis:  Two  groups  of  ad-hoc  communicators  emerged  on  the  UNITE 
E-mail  system.  The  larger  group  was  fairly  heterogeneous  while  the  smaller  was 
entirely  made  up  of  librarians. 

ttROIJP  ONE  (N=14)  OROIIP  TWO  (N  =  5) 

Science  5  Library/Ed  Comm  5 

University/ Adult  4 

Gen'l  Elementary  3 

Library/Ed  Comm  1 

Other  1 

nrosstahulation:  A  content-area  by  content-area  chi-square  of  links  formed 
found  that  teaching  content  area  was  a  significant  constraint  on  communication. 
The  chi-square  value  (124.47,  df  =  30)  was  significant  at  the  p  <  .01  level.  The 
adjusted  residuals  for  the  cells  denoting  communication  by  librarians  with 
librarians,  and  elementary  teachers  with  elementary  teachers,  were  significant  at 
the  p  <  .05  level.  In  other  words,  librarians  formed  more  links  with  librarians  and 
elementary  teachers  formed  more  links  with  elementary  teachers  than  would  be 
predicted  by  chance. 

Discussion 

Message  sending,  the  emergence  of  informal  groups,  and  communication 
between  members  of  pre-existent  groups  all  indicate  that  librarians  were  at  the 
center  of  electronic  messaging  on  the  UNITE  system,  and  most  communication 


ERLC 


203 


occurred  between  teachers  who  came  from  the  same  teaching  content  area.  This 
study  does  not  provide  evidence  that  the  communication  networks  teachers  form  on 
an  electronic  mail  system  will  differ  significantly  from  teachers'  face-to-face 
communication  networks,  both  being  dominated  by  contacts  with  other  teachers  in 
the  same  content  area.  Networking  itself  did  not  appear  to  encourage  an  increased 
volume  and  richness  of  information  flow  between  teaching  content  areas.  To 
accomplish  that  goal,  deeper,  underlying  strategies  to  encourage  cross- 
disciplinary  communication  appear  to  be  needed. 

As  a  basis  for  future  studies  to  establish  appropriate  strategies  for  improving  the 
educational  communication  process,  it  is  useful  to  look  at  three  factors  that 
emerged  from  the  network  analysis  and  from  teachers'  responses  to  open-ended 
questions  about  their  use  of  the  UNITE  E-mail  system. 

Time:  From  open-ended  responses  to  the  questionnaire,  it  was  apparent  that 
constraints  on  teachers'  time  put  constraints  on  their  use  of  the  system,  even  though 
the  system's  dynamic  resources  component  is  designed  to  save  teachers  time  and 
effort  in  the  long  run.  Overall,  teachers  and  librarians  expressed  frustration  with 
the  lack  of  time  they  had  to  take  care  of  their  normal  duties,  let  alone  familiarize 
themselves  with  the  computer  system.  Many  simply  did  not  see  what  they  would  get 
in  return  for  time  and  effort  invested  in  using  UNITE.  Librarians  have  greater 
leeway  in  how  they  structure  their  time,  with  less  of  it  devoted  to  direct,  disciplinary 
supervision  of  classes,  and  thus  were  better  able  to  devote  attention  to  UNITE  than 
were  classroom  teachers.  This,  in  part,  explains  the  higher  levels  of  use  of  the 
E-mail  system  by  librarians  than  other  teachers.  They  simply  had  more  time 
available  for  attending  to  it. 

Accessibility:  This  study  was  consistent  with  other  research  that  showed  that  the 
physical  accessibility  of  a  terminal  is  a  m^jor  determinant  in  who  will  use  it 
(West,  Inghilleri,  McSwiney,  Sayers,  &  Stroud  1989;  Steinfield,  1986).  The  farther 
away  the  terminal,  the  fewer  the  overt  opportunities  to  use  it,  and  in  some  broader 
sense  the  less  the  encouragement  to  use  it.  The  placement  of  shared  hardware  and 
resources  such  as  UNITE  terminals  is  clearly  a  major  issue  in  designing 
instructional  support  systems. 

Librarians  again  benefited  from  the  way  UNITE  was  implemented  in  the 
schools.  Many  UNITE  terminals  were  located  in  school  libraries  as  a  compromise 
to  allow  as  many  teachers  as  possible  to  get  relatively  equal  access,  and  a  few  others 
were  located  in  departmental  offices  or  elsewhere  in  the  school  building. 

ftoutine:  A  clear  factor  indicated  by  librarians'  responses  to  open-ended 
questions  was  the  degree  to  which  networking  on  UNITE  fit  easily  into  their  daily 
routines,  above  and  beyond  the  above-mentioned  factors  of  time  and  accessibility. 
Several  librarians  (and  other  teachers)  pointed  out  that  librarians  normally  keep 
in  touch  with  other  librarians  on  a  fairly  regular  basis  to  help  locate  and  share 
instructional  materials.  The  most  common  example  was  phoning  around  to  locate 
books  to  request  for  interlibrary  loan.  Much  of  this  activity  translated  directly  to 
E-mail;  E-mail  probably  encouraged  an  increase  in  communication  in  general 
among  librarians  because  they  were  already  in  the  habit  of  networking  and 
UNITE  gave  them  another  mechanism  for  doing  it.  Librarians  were  among  the 
early  users  of  UNITE  and  some  of  them  were  the  system's  biggest  boosters. 

For  other  teachers,  electronic  mail  did  not  fit  into  the  daily  routine  with  the 
same  ease.  While  nearly  all  of  tho  teachers  who  participated  in  this  study  were 
computer  literate,  hardly  any  of  them  indicated  that  they  used  computers  for  word 
processing,  figuring  grades,  or  other  daily  instructional  management  tasks.  They 
had  no  computer  on  their  desk.  UNITE  represented  a  form  of  computer  that  was 
"over  there  somewhere,"  not  part  of  what  they  normally  did  to  prepare  lessons.  The 
daily  routine  does  not  yot  include  the  use  of  computers  for  familiar  tasks,  so  the  use 
of  computers  for  relatively  novel  tasks  remains  unexplored.  Ono  of  those 
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unexplored  tasks  is  the  use  of  computer-mediated  communication  for  expanding 
teachers'  personal  networks  beyond  their  usual  spheres  to  include  contacts  from 
varying  disciplines  with  potentially  novel  approaches  to  instructional  tasks  and 
problems. 

In  some  way  all  three  factors  -  time,  accessibility,  and  routine  discouraged 
adoption  of  computer-mediated  communication  by  teachers  in  general  at  the  time  of 
this  study,  while  at  the  same  time  the  use  of  the  system  by  librarians  demonstrated 
its  promise  and  usefulness.  At  such  time  as  teachers  do  become  routinely  familiar 
with  computer  use,  electronic  mail  could  play  an  expanding  role  in  educational 
communication,  and  it  may  well  yet  encourage  communication  between  teachers 
in  different  teaching  content  areas,  creating  a  genuinely  new  and  optimally  useful 
set  of  professional  contacts.  But  the  introduction  of  computers  for  E-mail  and  other 
electornic  communication  in  schools  by  itself  may  be  insufficient  to  encourage 
cross-disciplinary  communication.  Specific  strategies  should  be  carefully 
considered  prior  to  implementation  of  any  electronic  mail  or  other  CMC  system  for 
it  to  reach  its  full  potential  in  enriching  the  communication  environment  of  the 
school. 
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Conducting  Qualitative  Research  in  Instructional  Teclinology:    Methods  and 


Various  philosophical  orientations  to  interpretive  research  are  categorized 
under  the  umbrella  term  qualitative  research.  For  example,  etlwography, 
phenomenology,  interpretive  research,  symbolic  uiteractionism,  and  ecological 
psychology,  to  identify  a  f*w  categories,  are  all  orientations  of  qualitative  research 
(Glesne  &  Peahkin,  1992).   All  forma  of  qualitative  research  share  common 
characteristics  and  research  methods.  It  is  beyond  the  scope  of  tliis  paper  to  discuss 
all  categories  of  qualitative  research   The  focus  of  this  paper  is  to  initiate  the 
novice  researcher  to  the  case  study  and  the  use  of  ethnography  and 
phenomenological  methods  and  techniques  of  research. 

Ethnography 

Both  ethnographic  and  phenomenological  research  are  interpretive  / 
descriptive  forms  of  research.  At  the  center  of  ethnographic  research  is  the  concept 
of  culture  (Tesch,  1988).  Goetz  and  LeCompte  ( 1984)  describe  ethnography  as 
"analytic  descriptions  or  reconstruction's  of  intact  cultural  scenes  and  groups" 
(p.  2).  The  ethnographic  investigation  focuses  upon  the  social  organization  of  a 
group  in  order  to  examine  cultural  processes  and  perspectives  of  those  within  the 
culture.  Hie  meaning  people  give  within  a  culture  to  their  experiences  is  not 
accidental  to  the  ethnographer.  Rather,  the  meaning  assigned  to  experiences  and 
behaviors  displayed  by  those  in  a  culture,  are  the  results  of  a  complex  mixture 
among  objects,  situations,  and  events. 

Applying  ethnographic  research  to  education  necessitates  recognizing 
educational  «*.  Hangs  as  cultural  entities.  Social  scientists  believe  that  human 
behavior  is  significantly  influenced  by  the  environment  in  wliich  it  occurs. 
Ecological  psychologists  clrim  that  if  one  ultimately  wants  to  generalize  research 
findings  to  the  everyday  world,  then  the  research  must  be  conducted  in  the 
environment  in  which  the  behavior  occurs  and  everyday  forces  are  recognized.  As 
Wilson  observes,  H  The  inability  of  classical  learning  theories  to  say  very  much 
that  is  meaningful  about  everyday  classroom  learning  can  be  explained  in  part  by 
the  absence  of  these  school/organizational  forces  in  the  research  laboratories  where 
the  theories  were  developed  (Wilson,  1977,  p.  248). 

Ethnographic  design  requires  investigative  strategies  wliich  enable 
cultural  reconstruction.    Ethnographic  strategies  are  empirical  and  naturalistic. 
Participant  and  non  participant  observation  are  UBed  to  obtain  firsthand,  sensory 
accounts  of  phenomena  as  they  occur  in  natural  settings,  in  addition,  strategies 
arc  used  to  elicit  phenomenological  data,  wliich  represent  the  view  of  the 
participants  being  investigated.    Oiaracteristics  of  classical  educational 
ethnography  strategies  include: 

".  .  the  investigation  of  a  small,  relatively  homogeneous  and 
geographically  bounded  study  site  .  .  .  long  term  and  repeated  residence  of 
the  researcher  at  the  site  ...  use  of  participant  observation  as  the  preferred 
data  collection  strategy,  supplemented  with  a  variety  of  ancillary 
techniques  ...  a  data  base  consisting  primarily  of  field  notes  .  .  .  and  ...  a 
preoccupation  and  explanation  of  the  culture,  life  ways,  and  social  structure 
of  the  group  under  investigation.  .  .  (Goetz  and  LeCompte,  1984,  p.  17)'. 

Phenomenology 

At  the  center  of  phenoinenologicaJ  research  is  the  human  experience  of  an 
occurrence.  Phenomenology  asks  the  question,  wltnt  makes  somiiUung  what  it  is 
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(Vhii  Maanen,  1U4M3)    Phenomenological  ivsoanJiorii  believe  Uiat  tiiere  are 
liuihiplo  wayn  of  interpreting  event*  for  each  |>erson,  and  Uiat  a  person's 
Interpolation  is  wliat  constitutes  reality  for  Uiat  person  (Bogdan  &  Biklen,  1982). 
What  distinguishes  phenomenology  from  other  qualitative  research  approaches  is 
that  Uie  "subjective  experience  in  at  the  neuter  of  the  Utqulry"  (Tesch,  1988,  p.  2). 

'Hie  phenomenological  research  tradition  relies  on  two  assumptions:  TTiat 
I Ikt  pcm;plJous  of  the  person  being  interviewed  are  valid;  and  that  the  experience 
mikI  background  of  the  researcher  are  sufficient  to  analyze  the  findings  in  a 
scholarly  ami  responsible  maimer.    Phenoinenological  researchers  depend  almost 
exclusively  on  in-depth  interviews  as  their  main  investigative  strategies,  using 
observation  only  where  verbalization  is  inadequate  (Teach,  1988). 

Phenomenology  does  not  rely  on  the  "thick  or  rich"  data  that  is  generally 
associated  with  Uie  ethnographic  study.  Instead,  data  are  collected  from  the 
experiences  of  Uie  participant  and  derive  meaning,  because  of  and  in  conjunction 
with,  the  background  and  experience  of  the  researcher.  Van  Maanen  (1990) 
connects  the  characteristics  of  thoughtfulness  and  love  to  the  interview  process. 
These  characteristics  enable  the  researcher  to  understand  how  the  participant  lived 
Uie  experience  that  is  being  described.  Hie  researcher  and  Uie  participant  come  to 
an  understanding  of  the  experience.  Teseh  (1988)  describes  the  phenomenological 
interview  as  "dialogical  reflection"  rather  than  a  set  of  questions  and  answers. 
Participants  are  encouraged  to  reiterate  and  express  themselves  in  different  ways, 
in  an  attempt  to  provide  the  researcher  wiUi  an  understanding  of  what  constitutes 
the  participants'  reality  of  an  event  or  interaction. 

Case  Study 

There  is  little  agreement  as  to  the  exact  definition  of  a  case  study.  Case 
studies  provide  a  holistic  description  of  an  environment,  They  convey  to  the  reader 
a  Bense  of  being  there,  or  a  vicarious  experience    Lincoln  and  Ouba  (1985)  propose 
that  case  studies  may  be  composed  for  dillhront  purposes  and  written  at  different 
analytical  levels.  Hieir  suggestions  of  categories  lor  case  studies  and  brief 
definitions  of  Uie  categories  are  as  follows  (1)  chronicle  (to  record  temporally);  (2) 
render  (to  provide  vicarious  experience),  (ill  teach  (as  providing  instructional 
materials),  and  (4)  test  (to  test  certain  Uioories  hiuI  hypotheses  in  a  setting). 
However,  it  is  recognized  that  any  case  may  serve  multiple  purposes  and  tit  under 
more  tiuui  one  category. 

Impending  on  the  piu^osc  of  Mm  mse  Htildy,  Lincoln  and  Utiha  propose  the 
study  may  be  written  at  different  analytical  levels,  Mtid  deluding  on  purpose  of  the 
study,  insult  in  different  phidurtH    The  analytical  levels  they  recognize  include,  " 
a  merely  factual  level,  an  luterpintutivc  level,  mid  mi  evaluative  level  (Lincoln 
and  (luba  1985,  p  301 )"  Due  to  Uie  evolving  nattuv  el  a  naturalistic  study,  tiiey 
suggest  Uiat  most  naturalistic  researchers  will  want  to  chronicle  aiul  hinder  at  the 
factual  level,  engage  In  iuterpivtation  tor  research  and  engage  III  evaluation  for 
policy  analysis. 

Research  Questions 

r11ie  research  questions  in  u  qualitative  study  may  be  vague  at  tiie  onset  of 
the  study   Performing  a  pilot  study  helps  Uie  researcher  clarify  Uie  research 
questions.  It  provides  an  opportunity  for  Uie  researcher  to  explore  the  setting, 
clarify  questions  and  learn  how  to  be  a  part,  of  Uie  setting  (Ulesne  &  Peshkin,  1992). 
Additionally,  a  thorough  acquaintance  wiUi  the  related  research  and  Uieory  helps 
the  researcher  to  focus  Uie  original  research  questions  (Wilson,  1977). 

Research  objectives  formulated  from  Uie  research  questions  can  help  guide 
the  data  collection,  however,  the  researcher  ivinaiiui  free  to  explore  new  areas  or 
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other  areaB  as  they  arise  out  of  the  researchers  observations.  The  qualitative 
researcher  is  always  aware  that  new  information  may  change  the  research 
questions  or  create  a  new  question.  Once  in  the  setting  it  may  become  apparent  that 
the  initial  research  questions  are  insufficient  or  inappropriate,  resulting  in  new 
questions  and  direction  of  the  study. 

Population 

Purposeful  selection  best  describes  the  selection  of  participants  in  a 
qualitative  study.  Participants  are  identified  and  selected  by  their  behaviors  and 
beliefs,  as  interpreted  by  the  researcher,  as  to  their  relevance  in  exploring  the 
research  questions.  It  is  impossible  for  the  researcher  to  know,  at  the  onset  of  the 
study,  all  of  those  who  will  eventually  become  participants  in  the  study. 
Identification  of  participants  evolves  as  does  the  study  (Glesne  &  Peshkiu,  1992, 
Lincoln  &  (luba,  1985) 

Data  Collection  and  Analysis 

Rich  data  collected  from  the  research  setting  provides  an  illuminating 
understanding  of  classroom  events  and  generates  additional  ideas  and  questions 
concerning  teachers  and  technology  (Shoenfield  &  Verban,1988).   Tlie  instrument 
for  data  collection  in  ethnography  and  phenomenological  research  is  the  human 
being.  Hie  goal  of  the  researcher  is  to  understand  the  research  setting,  the 
participants  and  their  behavior.  The  researcher  recognizes  that  it  is  not  enough  to 
simply  be  aware  of  the  framework  within  which  human  behaviors  occur,  but,  that 
there  must  be  a  systematic  effort  to  develop  an  understanding  of  the  context  in 
which  the  behaviors  occur. 

Data  collection  techniques  include  interview,  observation,  document  and 
record  analysis,  and  nonverbal  cues  (Lincoln  &  Guba,  1985).  Hie  researcher 
makes  calculated  decisions  as  to  what  kind  of  data  to  collect  and  whether  active 
probing  rather  than  simple  naturalistic  observation  should  be  used  (Wilson,  1977). 
For  example,  the  researcher  may  observe  an  interaction  between  two  students  and 
interpret  the  interaction  based  on  previous  observations  and  knowledge  of  the 
situation.  Or,  the  researcher  may  not  feel  that  a  true  interpretation  of  the  event  may 
be  made  without  further  probing  and  decides  to  interview  the  participants  to  gain  a 
better  understanding  of  the  Incident 

Handwritten  field  notes  are  a  common  method  of  recording  the 
researchers  observations  and  serve  to  keep  the  researcher  attentive.  While  other 
means  of  recording  observations  are  available,  such  as  the  camcorder  or  tape 
recorder,  the  researcher  must  be  aware  of  the  potential  of  these  devices  to  influence 
the  research  setting.  Participants  may  not  be  as  open  to  the  researcher  if  they  feel 
threatened  by  a  permanent  recording  (Lincoln  &  Uuba,  1985) 

Field  notes  consist  of  detailed  descriptions  of  the  setting,  events,  and 
interactions  of  participants.  They  also  provide  a  place  to  record  the  researclior's 
ideas,  reflections  and  interpretations  of  the  observed  events.  Notes  written  in  the 
field  may  initially  be  only  a  few  hastily  written  words  to  help  the  researcher 
remember  exact  descriptions,  that  will  later  be  expanded,  or  they  may  be  full 
detailed  descriptions  of  the  event  (Olesne  &  Peshkin,  1992). 

The  format  of  field  notes  is  dependent  upon  the  individual  preferences  of  the 
researcher.  Some  researchers  prefer  loose  leaf  notebooks  or  index  cards  to  record 
observations.  This  enables  the  researcher  to  organize  observations  into  categories 
as  the  study  progresses  rattier  than  waiting  until  the  end  of  data  collection  to  begin 
categorizing.  Other  researcher  prefer  to  remtil  their  observations  in  spiral 
notebooks  ami  later  transcribe  them  to  computer  files    Researchers  follow  various 
methods  of  coding  and  organizing  their  field  notes    Coding  may  be  according  to 
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chronological  order,  audi  as  historical  information  and  events,  or  follow  themes 
tliat  have  emerged  during  the  collection  process 

Documents  provide  additional  sources  of  data  for  the  researcher.  Lincoln 
and  Guba  (1985),  suggest  the  following  reasons  for  the  inclusion  of  documents  as  a 
data  resource:  (1)  availability  and  low  cost;  (2)  a  stable  source  of  information 
which  can  be  analyzed  and  re  analyzed,  (3)  they  are  contextually  relevant  and 
written  in  the  natural  language  of  the  setting,  (4)  they  are  often  statements  that 
satisfy  some  accountability  requirement,  and  (ft)  they  are  lion  reactive,  unlike 
human  respondents.  However,  they  also  caution  tliat  records  can  be  hi  error,  either 
intentionally  or  unintentionally 

Additional  sources  wliich  have  the  potential  to  provide  a  rich  source  of  data 
are  informants  and  found  pictures  Informants  may  play  a  variety  of  roles  in  the 
research  setting  and  are  invaluable  to  the  researcher  in  a  closed  community. 
Their  membership  in  the  researched  group  who  may  allow  them  to  direct  the 
researcher  to  unexplored  data  resources,  provide  introductions  to  key  members  of 
the  group,  or  even  collaborate  with  the  researcher  throughout  all  pliases  of  the  study 
((Mesne  &  Pushkin,  1992) 

Pict  ures,  either  still  or  motion,  can  provide  in  depth  detail  about  a  setting  or 
event   However,  care  should  Imj  taken  in  interpreting  the  events  depicted  in  a 
picture    Pictums  found  without  accompanying  dialogue  may  be  misinterpreted. 
Additionally,  photography  and  filming  techniques  can  be  used  to  cast  an  event  in  a 
particular  mode. 

The  length  of  time  in  the  field  interacts  with  the  conceptual  perspective  the 
researcher  takes.  For  example,  the  process  of  time  or  what  happened  in  the  setting 
during  a  specific  frame  of  time  may  be  of  specific  interest  to  the  researcher.  In  the 
school  setting  the  period  of  time  of  a  study  may  be  a  semester,  school  year,  grading 
period,  the  length  of  a  project  ( Smith,  1979).  Observations  of  the  setting  should  not  be 
limited  to  one  specific  day  of  the  week  or  time  of  day,  during  the  study.  They  should 
be  made  at  various  times  throughout  the  study  in  relation  to  events.  For  example, 
the  activities  in  a  school  setting  will  not  be  the  same  on  the  day  before  a  school 
holiday  «s  tlie  day  before  the  administering  of  a  national  test 

Triangulation  improves  the  probability  that  interpretations  of  data  will  be 
credible.  Tliis  practice  involves  the  incorporation  of  multiple  data  sources, 
methods,  investigators  and  theoretical  perspectives  in  the  study  (Glesne  & 
Peshkin,  1992,  Lincoln  &  Guba,  1985). 

I)ata  analysis  is  the  researcher  s  attempt  to  derive  meaning  or  information 
from  the  data  collected.  Data  collection  and  data  analysis  performed 
simultaneously  enables  the  researcher  to  focus  and  direct  the  study.  Glesne  and 
Peahkin  (1992)  emphasize  that  analysis  is  a  continuing  process,  rather  than  a 
specific  stage,  that  should  commence  as  soon  as  the  study  begins. 

Them  am  various  strategies  for  analyzing  data.  Most  researchers  find  it 
useful  to  look  lor  themes,  strands,  incidents  and  commonalties  in  the  data  as  it  is 
collected.    Developing  metaphors  and  analogies  may  help  the  researcher  analyze 
the  data  To  equate  a  situation  to  another  situation,  may  help  the  researcher  develop 
an  awareness  that  would  not  have  been  apparent  otherwise  (Bogdan  &  Biklen, 
1982)    For  example,  institutional  relationships  between  various  divisions  at  a 
southwestern  university  were  compared  to  a  marriage  separation  by  Cashman 
(1992)  and  helped  to  piovide  the  researcher  with  understanding  in  a  complex 
situation 

Reporting  Results 
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in  writing  the  results  it  is  important  to  consider  all  interactions  between 
subjects,  objects,  environment,  ciUture,  and  belief  system.  Additionally,  prior 
experiences  and  theoretical  orientation  of  the  researcher  should  be  addressed. 

Van  Maanen  (1988)  recognizes  three  different  types  of  styles  or  tales  as 
forms  of  reporting  ethnographic  research:  realist,  confessional,  and 
impressionist  Hie  first  style  is  the  realist  tale,  in  which  the  author  takes  an 
omnipotent  view.  In  this  style  tlie  author  is  concerned  with  the  minute  details  of  the 
participants'  lives.  These  details  are  presented  in  an  authoritarian  style  that 
seems  to  declare  the  authors'  viewpoint  as  the  reality. 

The  second  style,  is  the  confessional  tale  in  wliich  the  author  is  present 
tliroughout  the  text.  In  the  confessional  tale  authors  recognize  themselves  an 
human  beings  who  are  prone  to  mistakes,  but  eventually  arrive  at  a  new 
understanding. 

The  third  style,  is  the  impressionist  tale  which  makes  use  of  dramatic 
recall  in  which  the  reader  experiences  vicariously  the  experiences  of  the  researcher 
in  the  field.  Rather  than  events  being  interpreted  by  the  researcher,  the  readers 
derive  their  own  interpretations.   Impressionist  tales,  according  to  Van  Maanen, 
"are  not  about  what  usually  happens  but  about  what  rarely  happens.  Tliese  are  the 
tales  that  presumably  mark  and  make  memorable  the  fieldwork  experience"  (Van 
Maanen  1988,  p.  102). 

Even  though  these  styles  of  reporting  are  derived  from  etlinograpliic 
writing,  they  are  styles  that  can  be  utilized  in  various  areas  of  qualitative  writing. 
It  is  the  decision  of  the  researcher  to  decide  which  reporting  style  or  combination  of 
styles  will  best  represent  the  study.  Additionally,  it  is  recognized  tliat  qualitative 
reporting  is  not  constrained  by  rules  of  style,  but,  is  enhanced  by  the  freedom 
allowed  the  researcher  to  determine  the  best  fonnat  of  reporting. 
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Learning  from  video: 
The  Influence  of  preconceptlone  on  Inveeted  mental  effort 

The  fact  that  those  who  frequently  watch  television  are  referred  to  as  "couch 
potatoes"  indicates  that  American  society  accepts,  and  In  fact  expects,  viewers  to  adopt  a 
passive  stance  toward  television  viewing.  Although  a  passive  response  to  television  may 
be  beneficial  when  the  goal  of  the  Individual  Is  relaxation,  the  viewers1  preconceptions  of 
television  as  an  easy,  passive  medium  may  Interfere  with  their  learning  from  the 
medium,  and  thus  be  detrimental  to  the  use  of  television  and  videotapes  for  instruction. 

In  an  often-cited  study,  Salomon  (1984)  measured  learners1  preconceptions  of 
the  ease  or  difficulty  of  print  and  television,  then  the  learners  received  either  a  print- 
based  or  video-based  lesson.  Upon  completion  of  the  lesson,  learners  completed  a 
questionnaire  on  the  amount  of  mental  effort  that  they  perceived  they  Invested  in  the 
learning  task,  and  then  were  given  an  achievement  test.  Salomon  found  that  students 
rated  television  as  easier  than  print.  Learners  In  the  print  condition  reported  investing 
more  mental  effort  and  had  higher  achievement  scores  than  students  who  received  the 
video  lesson.  Salomon  concluded  that  the  learners1  preconceptions  of  television  as  an 
easy  medium  influenced  the  mental  effort  expended  In  processing  the  television  lesson. 

In  Kozma's  (1991)  review  of  the  literature  on  learning  with  media,  he  points  to 
gaps  in  the  research  on  learning  from  television  and  implies  the  need  for  the 
development  of  a  theoretical  model  of  television  learning.  As  an  initial  step  in  the 
process,  and  toward  that  end,  this  paper  explores  the  theoretical  and  research  basis  for 
the  relationship  between  learners'  preconceptions  of  television  and  the  amount  of  mental 
effort  invested  in  learning  from  a  video-based  lesson. 

Salomon  (1983)  defines  the  construct  of  "mental  effort "  as  the  "number  of  non- 
automatic  elaborations  applied  to  a  unit  of  material"  (p.  42).  The  conscious,  purposeful 
processing  that  results  from  Increased  mental  effort  is  assumed  to  create  greater 
activation  of  the  learner's  mental  schemata.   According  to  information  processing 
theory,  a  schema  is  an  organized  network  of  knowledge  that  includes  concepts,  facts, 
skills,  and  action  sequences  organized  in  such  a  way  that  the  individual  elements  can  be 
stored  and  retrieved  in  terms  of  a  more  inclusive  concept  (Gagn6  &  Glaser,  1987).  As 
new  Information  is  received,  the  new  Information  cues  the  retrieval  of  related  prior 
knowledge  stored  in  the  learner's  schemata.  Through  the  process  of  elaboration,  the 
learner  makes  connections  between  the  new  information  and  information  retrieved  from 
prior  knowledge  (E.  Gagn6,  1985).  The  increased  contact  with  the  learner's  mental 
schemata  that  results  from  the  conscious,  non-automatic  generation  ot  elaborations,  or 
mental  effort,  Is  presumed  to  facilitate  the  retention  and  retrieval  of  the  new  material. 
In  contrast  to  automatic  processing  that  is  fast  and  effortless,  non-automatic  processing 
Is  deliberate,  conscious,  and  very  much  under  the  control  of  the  Individual.  Salomon's 
(1983b;  1984;  Salomon  &  Leigh,  1984)  findings  of  a  significantly  correlation  between 
the  amount  of  mental  effort  learners  reported  Investing  In  a  lesson  and  learners' 
preconceptions  of  the  medium  of  presentation  suggest  that  preconceptions  of  the 
processing  requirements  of  video  may  influence  the  actual  mental  effort  expended  in 
learning  from  a  video-based  lesson. 

This  paper  presents  a  model  of  the  influence  of  learners1  preconceptions  on  the 
amount  of  mental  effort  Invested  In  learning  from  television.  The  proposed  model 
suggests  that  learners'  preconceptions  are  Influenced  by  the  characteristics  of  the 
media,  the  characteristics  of  the  task,  and  the  characteristics  of  the  learner.  The 
following  sections  will  review  the  theoretical  and  research  basis  of  these  proposed 
relationships,  present  the  practical  Implications  of  the  current  evidence  in  the  area,  and 
suggest  an  agenda  for  further  research. 
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Insert  Figure  1  about  here 


Preconceptions  of  Media 

Weiner  (1985)  proposed  that  learners'  perceptions  of  the  ease  or  difficulty  of 
learning  from  a  particular  medium  may  Influence  the  amount  of  effort  invested  In  the 
learning  task.  Weiner  contended  that  Individuals  engage  in  tasks  in  order  to  gain 
information  about  their  ability.  Individuals  seem  to  work  best  at  tasks  of  intermediate 
difficulty  because  engaging  in  difficult  tasks  provides  little  information  about  their 
ability  and  may  be  perceived  as  a  waste  of  effort,  while  easy  tasks  also  provide  little 
information  about  their  ability  and  are  perceived  as  unnecessary  for  success.  When 
faced  with  tasks  that  learners  perceive  to  be  of  intermediate  difficulty,  some  individuals 
may  exert  a  maximum  of  effort.  This  theory  may  account  for  Salomon's  (1983b;  1984; 
Salomon  &  Leigh,  1984)  findings  that  learners  reported  Investing  less  mental  effort  in 
learning  from  a  video-based  lesson  that  was  perceived  as  easy  than  from  a  print-based 
lesson  that  was  perceived  as  more  difficult  (than  the  video-based  lesson). 

The  results  of  Salomon  and  Leigh's  (1984)  investigation  of  the  responses  of  high 
and  low  ability  students  to  a  television  and  print  lesson  lends  credence  to  the  idea  that 
preconceptions  of  the  effort  required  to  process  a  lesson  influence  the  actual  effort 
expended  In  learning  from  the  lesson.  Salomon  and  Leigh  presented  learners  with  a 
television  and  print  version  of  a  story.  Consistent  with  earlier  findings,  they  found  that 
the  amount  of  mental  effort  reported  by  the  TV  group  was  significantly  lower  than  that 
reported  by  the  print  group.  In  addition,  high-ability  students  reported  investing 
significantly  less  effort  than  the  low-ability  students  in  processing  the  televised  story. 
Correspondingly,  the  high-ability  students  who  viewed  the  televised  version  recalled 
significantly  less  than  the  low-ability  students  who  viewed  the  same  TV  story  and 
significantly  less  than  the  high-ability  students  that  were  exposed  to  the  print  version. 
Weiner's  theory  suggests  that  the  low-ability  students  may  have  perceived  the  task  of 
learning  from  television  as  of  intermediate  difficulty  and  thus  exerted  a  maximum  of 
effort,  while  those  of  high-ability  may  have  perceived  the  televised  lesson  as  too  easy  to 
provide  them  with  additional  information  about  their  ability.  Weiner's  theory  suggests 
that  If  learners  can  come  to  view  television  as  worthy  of  the  exertion  of  additional 
mental  effort,  or  moderately  difficult,  then  they  will  exert  a  maximum  of  effort. 

Researchers  who  have  Investigated  the  nature  of  learners'  preconceptions  of 
video-based  materials  have  questioned  them  on  the  perceived  ease  or  difficulty  of 
learning  from  several  forms  of  media,  the  amount  of  effort  they  usually  invest  in 
processing  mediated  materials,  their  media  preferences,  and  their  perceptions  of  the 
potential  of  various  media  for  facilitating  learning.  Several  researchers  have  concluded 
students  in  grades  thfee  though  ten  perceive  television  to  be  significantly  HeaslerH  than 
print  (Krendl,  1986;  Salomon,  1983b;  1984;  Salomon  &  Leigh,  1984)  and  computers 
and  writing  (Krendl,  1986). 

However,  there  Is  some  evidence  that  learners'  preconceptions  of  media  vary 
with  the  age  and  experience  of  the  learner.  Using  methodology  that  was  similar  to  that 
used  by  Krendl  (1986),  Cambre  (1991)  found  that  the  media  preferences  of  Inservice 
teachers  differed  from  younger  students:  whereas  teachers  preferred  reading  and 
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watching  television  to  using  a  computer  and  writing,  students  in  grades  three  through 
ten  preferred  television  watching  and  using  a  computer  to  reading  and  writing.  Bordeauz 
and  Lange  (1991)  found  that  the  amount  of  effort  children  reported  investing  in 
children's  programs  decreased  as  the  age  of  the  viewer  Increased.  As  children  became 
older,  they  may  have  become  more  proficient  in  processing  children's  television 
programs  and,  thus,  reported  the  need  to  invest  less  effort  than  younger  children. 

Using  a  questionnaire  that  was  modeled  after  Salomon's  (1983;  1984), 
Cennamo,  Savenye  and  Smith  (1991)  also  found  evidence  that  age  and  experience  may 
Influence  learners'  preconceptions  of  the  ease  of  learning  from  a  medium.  They 
examined  the  preconceptions  of  pre-service  teachers  regarding  the  ease  of  learning  from 
several  forms  of  video-based  instruction.  The  data  analysis  indicated  that  television  was 
perceived  to  be  significantly  more  difficult  than  instructional  television  and  interactive 
video,  and  Instructional  television  was  perceived  to  be  significantly  more  difficult  than 
interactive  video.  Although  these  results  were  surprising  in  light  of  previous  findings 
regarding  the  perceived  ease  of  television  viewing,  the  past  academic  successes  of  the 
learners  and  their  major  area  of  study  may  have  provided  the  preservice  teachers  with 
greater  insight  into  the  learning  process  than  the  younger  students  who  participated  in 
earlier  studies. 

Kunkel  and  Kovaric  (1983)  provided  evidence  that  learners  may  have  different 
preconceptions  of  educational  materials  than  they  have  of  entertainment  materials. 
College  students  reported  investing  significantly  more  mental  effort  in  processing  a 
television  program  that  was  designed  for  a  Public  Broadcasting  Station  (PBS)  than  in 
processing  a  program  that  was  designed  for  a  commercial  television  station.   Kunkel  and 
Kovaric  reported  that  the  majority  of  PBS  programs  are  more  serious  and  educational 
than  commercial  television  programs,  and  the  results  of  this  study  suggest  that  the 
students'  perceptions  of  the  effort  required  by  the  two  types  of  programs  may  have  been 
influenced  by  an  awareness  of  this  difference. 

Several  researchers  suggest  that  the  content  of  the  television  program  may 
influence  learners'  perceptions  of  the  effort  required  to  process  the  program.  Salomon 
(1983)  questioned  college  students  on  the  amount  of  mental  effort  generally  expended  on 
televised  and  printed  adventure,  sports,  news,  science,  and  crime  stories.  He  found  that, 
with  the  exception  of  the  news,  the  estimated  amount  of  invested  mental  effort  was  lower 
with  television  than  with  print.  Using  Salomon's  questionnaire,  Beentjes  (1989)  found 
that  Dutch  children  did  not  always  perceive  it  to  be  easier  to  learn  from  television  than 
from  books;  instead,  he  found  that  students'  ratings  of  the  ease  or  difficulty  of  learning 
from  a  particular  medium  was  strongly  dependant  on  the  topic  in  question.  For  example, 
the  Dutch  children  perceived  that  it  was  easier  to  learn  soccer  rules  from  television 
than  from  a  book,  but  felt  that  it  was  easier  to  learn  to  solve  math  problems  from  a  book 
than  from  television. 

Cennamo  (1992)  investigated  learners'  preconceptions  of  the  ease  of  achieving 
various  learning  outcomes  (psychomotor,  affective,  verbal,  intellectual)  through  the 
media  of  interactive  video,  computers,  television  and  books.  The  results  indicated  that 
there  was  a  strong  interaction  between  the  presentation  medium  and  the  domain  of  the 
target  learning  outcome.  Like  the  sixth-grade  Dutch  students  who  participated  in 
Beentjes's  (1989)  study,  the  undergraduate  students  who  participated  in  this  study  did 
not  perceive  it  to  be  consistently  easier  to  learn  from  television  than  from  books. 
Although  television  was  rated  as  easier  than  books  for  learning  psychomotor  skills  and 
attitudes,  books  were  rated  as  the  easier  medium  for  learning  intellectual  skills  and 
verbal  information.  The  overall  means  for  television  and  computers  suggested  that  they 
were  perceived  as  very  similar  in  ease  of  learning;  however,  an  examination  of  the 
individual  means  for  each  domain  indicated  that  television  and  computers  were  believed 
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to  facilitate  the  ease  of  learning  very  different  types  of  skills.  Whereas  students 
perceived  it  to  be  easier  to  learn  attitudes  and  psychomotor  skills  from  television,  they 
perceived  it  to  be  easier  to  learn  intellectual  skills  and  verbal  information  from 
computers. 

In  general,  research  on  learners1  preconceptions  of  television  and  video  indicates 
that  learners  perceive  television  as  an  easy  medium,  they  feel  it  requires  little  mental 
effort,  and  they  believe  they  would  learn  little  from  a  video-based  lesson.  However, 
learners'  preconceptions  of  media  may  change  as  they  become  more  experienced. 
Learners*  preconceptions  of  the  ease  of  learning  from  television  also  may  vary  depending 
on  the  content  presented  through  the  medium. 

Mental  Effort 

Several  researchers  (Britton,  Muth,  &  Glynn,  1986)  have  noted  that  the  terms 
mental  effort,  attention,  concentration,  use  of  cognitive  capacity,  and  mental  workload 
all  refer  to  similar  concepts  and  refer  to  an  increase  in  the  cognitive  resources  devoted 
to  processing  the  stimulus.  Methods  of  assessing  mental  effort  and  similar  constructs 
fall  into  three  main  categories:  opinion  measures,  dual  tasks  techniques,  and 
physiological  measures.  Opinion  measures  encompass  the  variety  of  self-report 
measures  used  to  assess  mental  effort.  Opinion  measures  assume  that  the  investment  of 
effort  is  a  voluntary  process  which  is  under  the  control  of  the  individual,  and  as  such,  is 
available  for  introspection.   Dual  task  techniques  encompass  a  range  of  methods  which 
assign  the  learner  a  primary  task  such  as  reading  a  passage,  working  a  problem,  or 
viewing  a  videotape,  and  also  assign  the  learner  a  secondary  task  such  as  responding  to  a 
tone,  finger  tapping,  or  estimating  a  time  Internal.  Dual  task  techniques  assume  that 
there  is  a  limit  to  the  learner's  cognitive  capacity,  and  when  a  great  deal  of  cognitive 
capacity  is  consumed  by  the  primary  task,  there  will  be  less  capacity  left  to  devote  to  the 
secondary  task.  Dual  task  techniques  assume  that  the  differences  between  performance 
on  a  baseline  measurement  of  the  secondary  task  and  performance  under  experimental 
conditions  is  an  indication  of  the  amount  of  effort  expended  on  the  primary  task. 
Physiological  measures  are  based  on  the  assumption  that  there  Is  a  physiological 
response  to  increases  in  effort  expenditure.  The  difference  between  a  learner's  baseline 
measurement  of  the  physiological  process  and  a  measurement  taken  while  the  learner  is 
performing  some  task  is  assumed  to  be  reflective  of  the  amount  of  effort  the  learner  is 
investing  in  the  task. 

Researchers  have  investigated  factors  that  contribute  to  the  effort  Invested  in 
video-based  materials  using  a  variety  of  assessment  techniques  and  found  that  the 
amount  of  mental  effort  invested  in  a  video-based  lesson  can  be  Influenced  by  several 
motivational  and  cognitive  factors.  In  a  series  of  studies,  Salomon  (1983;  1984; 
Salomon  &  Leigh,  1984)  used  self-report  questionnaires  to  assess  the  amount  of  mental 
effort  that  learners  perceived  they  had  invested  in  processing  television  and  print-based 
lessons.  He  consistently  found  that  students  reported  investing  less  effort  In  processing 
a  video-taped  version  of  a  story  than  in  processing  a  text-based  version.  In  addition, 
students  who  were  instructed  to  learn  from  the  story  reported  investing  more  effort  in 
processing  the  story  than  those  who  were  instructed  to  read  or  view  the  story  for  fun 
(Krendl  &  Watklns,  1983;  Salomon  &  Leigh,  1984).  Salomon  concluded  that  learners' 
preconceptions  of  the  effort  required  by  television  and  print  and  the  perceived  purpose 
of  the  task  seemed  to  influence  the  effort  Invested  In  learning  from  the  television  or 
print-based  lesson. 
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Several  other  studies  suggest  that  the  complexity  of  video-based  materials  may 
influence  the  effort  expended  In  processing  a  lesson.  The  complexity  of  a  video-based 
lesson  may  be  caused  by  a)  perceptual  factors  such  as  the  simultaneous  presentation  of 
congruent  or  incongruent  audio  and  video  Information  or  b)  message-related  factors 
such  as  the  organization  of  the  content. 

Research  suggests  that  simultaneously  attending  to  auditory  and  visual  information 
requires  that  cognitive  resources  be  devoted  to  the  process.  Learners  Invest  more  effort 
in  processing  video-based  materials  that  present  information  through  both  the  auditory 
and  visual  channels  than  through  either  channel  alone  (Reeves,  Thorson,  &  Schleuder, 

1985)  .  In  addition,  learners  exert  more  effort  in  learning  from  materials  that  contain 
a  high  degree  of  correspondence  between  the  information  presented  through  the  audio  and 
video  channels  than  they  do  in  learning  from  materials  in  which  there  Is  little 
correspondence  between  the  video  and  auditory  information  or  in  which  there  is  only  a 
thematic  match  between  the  two  channels  of  information  (Grimes,  1990). 

Using  a  secondary  task  technique  to  assess  mental  effort,  Reeves  and  Thorson 
(1986)  found  that  cues  occurring  during  a  complex  moment  in  a  television  commercial 
elicited  longer  reaction  times  than  cues  that  occurred  during  less  complex  moments. 
Complexity  involved  both  auditory  elements  such  as  the  sentence  syntax  and  visual 
elements  such  as  pans,  zooms,  and  movement  within  the  scene.  Likewise,  Lang, 
Strickwerda,  and  Sumner  (1991)  found  that  learners  used  more  mental  effort  in 
processing  video  cuts  that  were  unrelated  to  the  prior  scene  than  in  processing  cuts 
which  were  related  to  the  previous  scene  through  visual  elements  or  narration.  The 
increases  in  effort  expended  during  complex  moments  such  as  these  may  result  from  the 
need  for  learners  to  search  their  memories  for  appropriate  schemata  in  which  to  store 
the  new  Information.  The  memory  search  required  to  make  sense  of  new  information 
may  account  for  the  increases  in  effort  documented  by  these  researchers. 

However,  videotaped  messages  that  contained  a  large  number  of  complex  elements 
produced  iaslfil  reaction  times  than  messages  that  were  less  complex  (Reeves  &  Thorson, 

1986)  .  These  findings  suggest  that  learners  expend  more  mental  effort  in  processing 
messages  that  are  simple,  overall,  than  in  processing  complex  messages.  The 
researchers  suggest  that  when  processing  complex  messages,  the  viewers  may  be  unable 
to  call  upon  appropriate  schemata.  Simple  messages  may  allow  the  viewers  to  easily  call 
upon  appropriate  schemata,  and  thus,  to  actively  engage  in  the  process  of  elaboration. 

The  influence  on  mental  effort  of  the  learners'  ability  to  call  upon  appropriate 
schemata  was  investigated  by  Meadowcroft  and  Reeves  (1989).  They  classified  children 
into  those  with  Hhigh  schema  development"  and  "low  schema  development".  Using  a 
secondary  task  technique  to  determine  the  amount  of  cognitive  resources  invested  in  the 
task,  these  researchers  presented  children  with  cartoons  in  a  jumbled  or  normal  form. 
Children  that  appeared  to  have  well  developed  schema  exerted  less  effort  in  processing 
the  content  of  the  programs  than  those  who  had  low  levels  of  schema  development.  It  is 
possible  that  children  with  high  schema  development  were  able  to  call  upon  an 
appropriate  schema  more  readily  and  to  fill  gaps  in  their  schemata  more  effectively  than 
those  with  less  well  developed  schemata.  The  effect  of  story  structure  was 
nonsignificant;  however,  all  students  allocated  more  effort  in  attending  to  information 
that  was  central  to  the  story  content  than  in  processing  other  peripheral  information. 
Meadowcroft  and  Reeves  concluded  that  children  in  the  high  schema  group  allocated  their 
cognitive  resources  strategically,  investing  more  effort  when  attending  to  a  scene  when 
it  was  central  to  the  story  content  that  to  scenes  that  were  incidental  to  the  story  content. 
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Mental  Effort  and  Achievement 

The  results  of  research  on  mental  effort  and  achievement  indicates  that  there  may 
not  be  a  linear  relationship  between  learners'  invested  mental  effort  and  their 
achievement  scores.  Under  certain  conditions,  increased  effort  may  result  in  increased 
achievement.  Yet,  under  other  conditions,  increased  effort  does  not  result  in 
achievement  gains. 

When  additional  resources  need  to  be  devoted  to  perceptual  processing  in  order  to 
"make  sense"  of  the  information,  increased  effort  does  not  always  result  in  increased 
achievement.  For  example,  Reeves,  Thorson,  and  Schleuc'er  (1985)  presented  learners 
with  a  videotaped  program  under  audio-only,  video-only,  or  audio-video  conditions  and 
found  that  reaction  times  to  a  secondary  task  were  significantly  delayed  for  those  who 
attended  to  the  audio-video  presentation.  However,  there  were  no  significant  differences 
in  learners*  achievement  scores  among  the  three  groups. 

In  other  studies,  increases  in  mental  effort  may  have  been  accounted  for  by  the 
effort  required  to  search  long-term  memory  for  appropriate  schemata  in  which  to  store 
the  new  information  and,  thus,  may  not  have  resulted  in  increased  achievement  scores. 
Lang,  Strickwerda,  and  Sumner  (1991)  found  that  learners  exerted  more  effort  in 
processing  video  segments  that  included  unrelated  cuts  representing  changes  in  content 
than  they  did  in  processing  segments  that  included  related  cuts  linking  information  to 
new  content  through  visual  elements  or  narration.  However,  learners  recalled  more  of 
the  information  surrounding  related  cuts  than  surrounding  unrelated  cuts.  It  is  possible 
that  when  the  learners  were  presented  with  an  unrelated  cut,  they  devoted  additional 
cognitive  resources  to  the  search  for  relevant  schemata  in  which  to  store  the  new 
information  and,  thus,  missed  the  content  necessary  to  perform  well  on  a  test  of 
information  surrounding  the  unrelated  cuts. 

Salomon  (1983b)  suggests  that  when  a  lesson  is  drastically  inconsistent  with 
learners'  schemata,  increased  mental  effort  is  not  sufficient  for  increased  achievement. 
He  found  that  students  who  viewed  a  scrambled  version  of  a  videotaped  program  reported 
investing  more  mental  effort  in  the  program  than  students  who  viewed  a  normal 
version;  however,  they  performed  poorer  than  the  students  who  viewed  the  normal 
program.  Salomon's  study  (1983)  suggests  that  simply  expending  more  effort  does  not 
guarantee  increased  achievement  when  learners  are  unable  to  create  a  coherent  mental 
model  of  the  content.  However,  when  Krendl  and  Watkins  (1983)  presented  learners 
with  a  videotape  which  included  irrelevant  segments  but  retained  the  original 
narrative,  learners  attempted  to  assign  meaning  to  extraneous  scenes  and  to  incorporate 
irrelevant  information  into  the  story  line.  In  Krendl  and  Watkins'  study,  the 
availability  of  a  main  intact  narrative  apparently  provided  the  structure  necessary  to 
tap  Into  the  viewer's  schemata.  In  Salomon's  study,  self  reports  of  increased  mental 
effort  indicated  that  the  learners  did  attempt  to  elaborate  on  the  material,  but  their 
lower  performance  suggests  that  they  may  have  been  unable  to  fit  information  into  their 
existing  schemata. 

However,  increased  effort  also  has  been  shown  to  result  in  increased 
achievement.  Grimes  (1990)  presented  learners  with  three  versions  of  a  videotaped 
program  and  found  that  learners'  reaction  times  to  a  secondary  task  were  longest  in  the 
condition  where  the  visual  and  auditory  channels  presented  corresponding  information 
(high  correspondence),  and  shortest  in  the  condition  where  there  was  only  a  thematic 
match  between  the  visual  and  the  audio  portion  of  the  program  (medium 
correspondence).  Scores  on  a  visual  recognition  test  followed  the  same  pattern;  learners 
who  viewed  the  program  with  high  correspondence  scored  highest  and  those  who  viewed 
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the  program  with  medium  correspondence  scored  lowest.  The  presence  of  auditory  and 
visua  information  with  a  high  degree  of  correspondence  may  have  provided  the  best 
conditions  under  which  to  elaborate  on  the  content,  thus  explaining  why  learners  who 
received  this  program  achieved  higher  test  scores. 

QammnfiT  fl^f no^rS  (Krendl  &  Wa,kins'  1983;  Salomon,  1983b;  Salomon,  1984; 
Salomon  &  Leigh,  1984)  suggest  that  the  benefits  of  increased  mental  effort  may  not  be 

fXSSf  °n  .a.,est  ?  ,ac,ual  recaNl  but  ins,ead  result  in  greater  inferences.  Holmes 
(1987)  expialns  that  "whereas  factual  recall  involves  understanding  explicitly  stated 
information,  inferences  require  the  reader  to  fill  in  missing  information  or  to  connect 
propositions  in  the  text  Implied  by  the  author"  (p.14).    Since  Salomon  (1981)  defines 
mental  effort  as  the  number  of  "non-automatic  elaborations"  applied  to  a  unit  of 
materia ,  and  the  inference  process  involves  conscious  elaborations  of  the  material  it 
seems  logical  that  Inference  questions  may  be  a  more  sensitive  measure  of  increased 
mental  effort  than  recall  or  recognition  tasks.   Using  self-report  questionnaires, 
r«  J«mv°?  (  9i3b:  1984;  *aio™on  &  Loigh,  1984)  and  Cennamo,  Savenye  and  Smith 
(1991)  found  a  significant  correlation  between  learners'  reported  mental  effort  and 
their  scores  on  a  test  of  inference  ilomr..  Learners'  who  reported  investing  a  large 
amount  of  mental  effort  received  hlghor  inference  test  scores  than  learners  who 
reported  Investing  less  effort. 


Influential  FactQf-s 

The  previous  section  described  the  proposed  relationships  among  learners' 
preconceptions  of  the  ease  or  difficulty  of  a  medium,  the  amount  of  mental  effort  they 
hvest  n  processing  a  lesson  using  that  medium,  and  their  resulting  achievement  scores 
Given  these  relationships,  the  real  questions  remain:  Why  do  learners  perceive  one 
medium  as  easier  or  more  difficult  than  another  for  learning  a  particular  lesson?  What 
can  be  done  to  overcome  learners'  perceptions  that  television  is  an  easy  medium 
requiring  very  little  investment  of  mental  effort?  What  factors  can  influence  learners 
to  Invest  the  optimum  amount  of  mental  effort  in  a  learning  task?  And  finally  how  can 
we  design  more  effective  video-based  materials? 

In  examining  the  literature  to  seek  answers  to  these  issues,  several  themes  seem 
to  emerge;  factors  which  Influence  learners'  preconceptions  of  the  effort  required  by 
video-based  materials  and  their  actual  or  perceived  mental  effort  expenditure  seem  to 

J2L        T lleS,01  charac,eris'ics  of  the  media,  characteristics  of  the  task,  and 
characteristics  of  the  learners. 
Charantorktins  nf  thft  fy|ftrj|a 

nmonn,SneVDeH«,C|lfr.af,eriS,icS  °'  vide°-based  'essons  have  been  shown  to  influence  the 
amount  of  effort  that  learners  perceive  is  needed  and  the  actual  effort  invested  in 
processing  a  video-based  lesson.  Several  researchers  have  shown  that  the  symbol 

fqflr41m^nyei  Q^hhe0mediUm  '^enoe  e,fort  (Reeves'  Thorson'  &  Schleuder, 
JSLi  t    i  .J         '  Salomon-  1984:  Sa|omon  &  Leigh,  1984).  The  perceptual 
complexity  of  the  scene  changes  (Lang,  Strickwerda,  &  Sumner,  1991;  Reeves  & 
,rSe'  122  and  ,n.e,de9ree  to  which  the  audio  and  video  elements  are  parallel 
SnronL!  ?,2  ^  '"J1"9"" Jhe  effort  squired  to  search  long-term  memory  for  an 
appropriate  schema  and  to  elaborate  on  the  content  of  the  lesson.  In  addition  effort 
investment  is  influenced  by  the  structure  of  the  video-based  lesson.   The  Importance  of 
a  segment  o  the  story  line  (Meadowcroft  &  Reeves,  1989)  influences  the  way  learners 
allocate  their  mental  effort  while  processing  a  video-based  lesson 
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Charactpristlcs  ot  the  Task 

Several  studies  have  varied  the  characteristics  of  the  task  in  order  to  investigate 
the  effects  on  learners'  preconceptions  of  the  effort  required  by  video-based  instruction 
and  the  amount  of  mental  effort  invested  in  such  a  task.  Learners  seem  to  exert  more 
mental  effort  in  a  lesson  from  which  they  are  told  to  learn  than  from  a  lesson  they  are 
told  to  view  for  fun  (Krendl  &  Watkins,  1983;  Salomon  &  Leigh,  1984).  Likewise, 
learners  report  investing  more  mental  effort  in  attending  to  educational  television 
programs  than  in  attending  to  commercial  television  programs  (Kunkel  &  Kovaric, 
1983).  The  domain  of  the  desired  learning  outcome  (Cennamo,  1992)  and  the  topic  of 
the  lesson  (Beent]es,1989)  also  seem  to  influence  learners'  preconceptions  of  the  effort 
required  by  video-based  instruction. 
Characteristics  of  the  Learner 

The  preconceptions  that  learners  bring  to  an  instructional  situation  regarding  the 
amount  of  effort  required  by  a  video-based  lesson  and  the  actual  amount  of  effort 
expended  in  processing  the  lesson  are  also  influenced  by  individual  characteristics  of  the 
learner.  Research  suggests  that  the  age  and  experience  of  the  learners  may  affect  their 
media  preferences  (Cambre,  1991),  the  amount  of  effort  they  report  investing  in 
viewing  a  television  program  (Bordeaux  &  Lange,  1991),  and  the  perceived  ease  of 
learning  from  a  video-based  lesson  (Cennamo,  Savenye,  &  Smith,  1991).  In  addition, 
there  is  other  research  which  suggests  learners'  attitudes  toward  a  medium  might  change 
as  they  gain  more  experience  with  that  medium  (Cambre,  1991;  Cennamo,  1992),  The 
amount  of  effort  invested  in  a  video-based  lesson  also  may  depend  on  the  intellectual 
ability  of  the  learner  (Salomon  &  Leigh,  1984). 

Summary 

This  paper  has  proposed  a  direct  relationship  between  learners'  preconceptions 
of  the  effort  required  to  process  a  video-based  lesson,  the  actual  mental  effort  expended 
in  processing  the  lesson,  and  learners'  scores  on  achievement  tests,  it  proposed  that 
learners  assess  a  learning  situation  in  terms  of  the  task  at  hand  and  the  medium  of 
presentation.  They  make  a  decision  as  to  the  ease  or  difficulty  of  the  task  based  on  their 
past  experience  with  similar  events.  They  decide  to  exert  a  certain  amount  of  effort  in 
processing  the  lesson.  If  characteristics  of  the  lesson  differ  from  those  expected  based 
on  past  experience  (i.  e.  harder  content,  more  complex  visually)  the  learners  reanalyze 
the  situation  and  make  "on-line"  modifications  of  their  effort  expenditures.  The  amount 
of  actual  effort  invested  in  the  task  may  be  influenced  by  cognitive  activities  such  as 
perceptual  processing,  searching  memory  for  appropriate  schemata,  or  elaborating  on 
the  content.  Increases  in  effort  which  result  from  the  elaboration  process  may  result  in 
increased  contact  with  the  learners'  mental  schemata  and  facilitate  retention  and 
retrieval  of  the  new  material.  There  are  several  practical  implications  of  the  proposed 
model,  yet,  there  are  numerous  questions  which  remain  to  be  answered.  The  following 
sections  will  present  practical  suggestions  which  arise  from  research  In  this  area  and 
suggest  areas  for  further  study. 
Practical  implications 

An  awareness  of  factors  which  affect  learners'  preconceptions  of  video  and  the 
amount  of  mental  effort  invested  in  video-based  materials  can  assist  instructional 
designers  in  creating  more  effective  video-based  lessons.  Techniques  as  simple  as 
informing  the  users  that  the  materials  are  designed  to  be  educational  (Kunkel  &  Kovaric, 
1983)  and  they  are  to  learn  as  much  as  possible  from  the  lesson  (Krendl  &  Watkins, 
1983;  Salomon  &  Leigh,  1984)  should  increase  the  effort  invested  in  learning  from  a 
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video-based  lesson.  In  addition,  designers  should  attempt  to  use  the  medium  of 
presentation  which  is  most  appropriate  for  the  domain  of  the  target  learning  outcome 
(Cennamo,  1992). 

The  challenge  for  instructional  designers  is  to  maximize  the  effort  that  learners 
expend  on  elaborating  the  content,  while  minimizing  the  effort  required  to  "make  sense" 
of  the  content  or  to  search  long-term  memory  for  an  appropriate  schemata  in  which  to 
store  the  new  material,  The  research  suggests  that  the  complexity  of  the  message  should 
be  reduced  through  the  use  of  simple  syntax  (Reeves  &  Thorson,  1986)  and  related  cuts 
(Lang,  et.  al.,  1991).  The  program  should  be  designed  so  that  there  is  a  high  degree  of 
correspondence  between  the  video  and  audio  portions  of  the  program  (Grimes.1990).  In 
addition,  the  Intended  message  should  be  central  to  the  story  content  (Meadowcroft  & 
Reeves,  1989).  Programs  which  provide  pauses  following  complex  elements  such  as 
unrelated  scene  changes,  videographics,  and  complex  sentences  may  provide  time  for  the 
viewers  to  search  their  long-term  memory  for  appropriate  schemata  before  additional 
critical  information  is  presented.  Music,  explanatory  examples,  and  other  mentally 
undemanding  content  may  be  used  to  allow  the  learners  time  to  prepare  to  elaborate  on 
new  content  Information. 

Although  the  instructional  designer  can  not  affect  the  age,  experience,  or  ability 
of  the  learners,  an  awareness  of  such  learner  characteristics  may  assist  the  designer  in 
creating  a  greater  match  between  characteristics  of  the  video-based  materials  and  the 
learner.  High-ability  learners  may  need  to  perceive  a  video-based  lesson  as  challenging 
in  order  to  perceive  it  as  necessary  to  gain  information  about  their  ability  (Salomon  & 
Leigh,  1984;  Welner,  1985). 
Topics  for  Further  Resflarrh 

It  has  been  suggested  that  learners'  preconceptions  of  the  effort  required  by  a 
learning  task  are  influenced  by  the  characteristics  of  the  media,  characteristics  of  the 
task,  and  characteristics  of  the  learner.  These  conclusions  are  based  on  learners'  self- 
reports  of  their  perceptions  of  media.  Future  research  needs  to  examine  the  reasons 
why  learners  perceive  one  medium  as  easier  or  more  difficult  than  another  for  learning 
a  particular  task.  In-depth  interviews  with  learners  regarding  the  reasons  why  they 
may  rate  one  medium  as  easier  or  more  difficult  than  another  may  provide  Insight  as  to 
other  factors  which  may  influence  their  preconceptions  of  the  effort  required  by  a 
particular  media-based  lesson. 

In  addition,  previous  investigations  of  the  amount  of  effort  invested  in  a  video- 
based  lesson  may  have  been  limited  by  the  methodology  available  to  assess  the  construct 
of  mental  effort.    Early  work  (Salomon,  1983;  Salomon,  1984;  Salomon  &  Leigh, 
1984)  used  self-report  questionnaires  to  document  the  learners'  effort  expenditures. 
Recently,  however,  researchers  (Beentjes,  1989;  Cennamo,  el.  al.,  1991)  have 
identified  a  need  for  more  precise  methods  of  assessing  mental  effort.  Validation  studies 
where  mental  effort  is  Investigated  using  a  variety  of  methods  may  yield  the  research 
tools  necessary  to  explore  other  techniques  which  may  increase  the  effort  expended  in 
processing  video-based  lessons. 

A  series  of  studies  In  human  factors  engineering  (Casali  &  Wlerwille,  1983' 
Wierwille  &  Connor,  1983;  Wierwille,  Rahimi,  &  Casali,  1985)  have  attempted  to 
determine  the  sensitivity  of  15  different  assessment  measures  for  detecting  variations 
m  the  workload  placed  upon  an  operator  of  a  flight  simulator  and  may  serve  as  a  model 
for  educational  researchers  Investigating  methods  of  assessing  mental  effort.  The 
researchers  found  that  the  assessment  measures  which  were  most  sensitive  to 
manipulations  in  the  operator  workload  varied  depending  on  the  type  of  task  performed 
by  the  operator.  For  oxamplo,  oyeblinks  and  eye  fixation  measures  wore  sensitive  to 
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increases  in  the  difficulty  of  a  cognitive  task,  but  were  not  sensitive  to  increases  in  the 
difficulty  of  a  communication  task.  Although  one  of  the  opinion  scales  (Modified  Cooper- 
Harper)  and  the  time  estimation  task  were  sensitive  to  increases  in  operator  demands 
across  perceptual  psychomotor,  communication,  and  cognitive  problem  solving  tasks, 
the  time  estimation  task  resulted  in  an  increase  in  the  error  rate  of  subjects  who  were 
performing  the  cognitive  task.  These  results  suggest  that  educational  researchers  who 
desire  to  assess  the  mental  effort  invested  in  learning  tasks  may  need  to  take  Into 
consideration  the  type  of  task  required  of  the  learner  and  the  possible  interference 
effects  of  the  assessment  measures  on  the  performance  of  the  primary  task. 

Future  research  needs  to  continue  investigating  techniques  that  may  be 
incorporated  Into  video-based  materials  in  order  to  Increase  the  amount  of  effort 
invested  in  processing  lessons  presented  through  this  medium.  Due  to  the  complexity  of 
the  symbol  systems  present  In  video-based  instruction  coupled  with  the  linear,  fixed 
pace  of  the  presentation,  it  may  be  beneficial  to  include  techniques  that  would  assist  the 
learner  in  actively  processing  the  content  in  a  more  efficient  manner.  Techniques 
designed  to  facilitate  the  learners'  search  for  appropriate  schemata  and  to  enhance  their 
abilities  to  elaborate  on  the  content  may  overcome  the  learners1  potential  conflict 
between  attending  to  new  Information  as  it  Is  being  presented,  conducting  a  memory 
search  to  activate  appropriate  schemata,  or  elaborating  on  the  content.  Techniques  that 
cue  the  learners  In  advance  as  to  the  appropriate  schema  to  engage  or  that  allow  the 
learners  extra  time  to  actively  process  new  information  may  increase  the  mental  effort 
that  learners  Invest  In  processing  a  video-based  lesson  and,  thus,  increase  the  amount  of 
Information  learned  from  the  lesson. 

Britton  and  his  colleagues  have  varied  the  structure  of  text-based  materials  to 
determine  the  effects  of  various  techniques  on  learners'  invested  mental  effort.  Using  a 
secondary  task  method  of  assessing  mental  effort,  they  found  that  the  amount  of  effort 
expended  in  reading  text  passages  was  influenced  by  the  meanlngfulness  of  the  passage 
(Britton,  Holdredge,  Curry,  &  Westbrook,  1979),  the  learners  prior  knowledge  of  the 
content  (Britton  &  Tesser,t982),  the  Importance  of  the  Information  (Britton,  Muth,  & 
Glynn,  1986),  the  style  of  writing  (Britton,  Graesser,  Glynn,  Hamilton,  &  Penland, 
1983),  embedded  questions  (Britton,  Piha,  Davis,  &  Wehausen,  1978),  text  syntax 
(Britton,  Glynn,  Meyer,  &  Penland,  1982)  and  the  presence  of  objectives  (Britton, 
Glynn,  Muth,  &  Penland,  1985).  It  may  be  useful  to  incorporate  similar  techniques  Into 
video-based  materials  to  determine  if  such  factors  influence  the  processing  of  text- 
based  and  video-based  materials  in  a  similar  manner. 

Future  research  also  should  investigate  the  effects  of  the  enhanced  processing 
capabilities  offered  by  computers  on  learners'  preconceptions  and  mental  effort.  The 
symbol  systems  available  for  books  and  television  are  very  similar  to  those  commonly 
available  via  computers  and  interactive  video  and  it  could  be  argued  that  interactive 
video  and  computer-based  instruction  are  sophisticated  variations  of  television  and 
books  with  enhanced  processing  capabilities.  Although  many  elaborate  techniques  for 
enhancing  Instruction  are  available  through  the  addition  of  the  computer,  computer- 
aided  instruction,  at  its  most  basic  level,  offers  computer-controlled  text  and 
Interactive  video  offers  computer-controlled  video. 

Finally,  the  proposed  link  between  learners'  preconceptions  of  the  effort 
required  by  a  medium,  the  actual  effort  expended,  and  learners'  achievement  scores  must 
be  investigated  In  a  systematic  fashion.  For  example,  the  domain  of  the  intended  learning 
outcome  seems  to  Influence  learners'  perceptions  of  the  ease  of  learning  from  a 
television  lesson  (Cennamo,  1992).  Learners  report  that  It  Is  easier  to  learn 
psychomotor  skills  and  attitudes  from  television  than  from  books,  but  that  It  is  easier  to 
learn  verbal  Information  and  intellectual  skills  from  books  than  from  television.  In 
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order  to  continue  an  investigation  of  the  proposed  relationships  among  learners' 
preconceptions,  mental  effort,  and  achievement,  future  research  should  investigate  a) 
whether  students  actually  invest  more  mental  effort  or  learn  more  when  instructed  in 
psychomotor  skills  and  attitudes  through  television  than  through  print-based 
Instruction,  and  b)  whether  learners  invest  more  mental  effort  in  learning  psychomotor 
skills  and  attitudes  from  television  than  in  learning  verbal  information  and  intellectual 
skills  from  television. 

This  review  of  the  literature  on  preconceptions  of  video-based  instruction  and 
mental  effort  has  also  Identified  several  areas  in  which  there  has  been  little  research. 
Future  research  should  investigate  the  influence  of  individuals'  learning  styles  on  their 
preconceptions  of  the  ease  of  learning  from  media,  the  amount  of  mental  effort  invested 
in  learning  from  a  video-based  lesson,  and  their  learning  achievement.  In  addition,  the 
reciprocal  relationship  between  learners'  experiences  of  success  or  failure  in  learning 
from  a  video-based  lesson  and  their  subsequent  perceptions  of  the  ease  or  difficulty  of 
learning  from  the  medium  in  the  future  should  be  investigated.    Salomon  (1981) 
reminds  us  that  mental  effort  has  motivational  as  well  as  cognitive  components; 
therefore,  future  research  should  incorporate  techniques  intended  to  increase  learners 
motivation  into  video-based  instruction  to  determine  the  effects  of  these  techniques  on 
mental  effort  and  achievement. 

This  paper  has  reviewed  the  current  research  literature  and  outlined  factors  that 
must  be  considered  in  future  investigations  of  learners'  preconceptions  of  video-based 
Instruction,  the  mental  effort  invested  in  learning  from  such  a  lesson,  and  techniques  to 
increase  the  effort  invested  in  processing  a  video-based  lesson.  The  results  of  this 
research  should  provide  Instructional  designers,  video  producers,  and  teachers  with 
practical  means  of  increasing  the  effort  that  learners  spend  in  processing  video-based 
materials  so  that  the  ultimate  goal  of  the  designer,  the  producer,  the  teacher,  and  the 
learner  can  be  met  as  efficiently  and  effectively  as  possible-  that  of  the  achievement  of 
the  desired  learning  outcomes. 
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Preconceptions  of  Mediated  Instruction 
for  Various  Domains  of  Learning  Outcomes 


Individuals  involved  in  teaching  and  training  are  confronted  with  a  vast  array  of 
potentially  beneficial  instructional  media.  Text,  television,  computers,  and  other  media 
all  promise  to  improve  learner  attention  and  achievement.  Several  researchers  (Krendl 
&  Watkins,  1983;  Merikoff,  1980;  Salomon,  1983;  Salomon  &  Leigh,  1984)  have 
suggested  that  learners'  preconceptions  of  media  are  an  influential  factor  in 
achievement.  For  example,  Salomon  (1984)  found  that  learners  perceive  television  as 
easier  than  books,  yet  they  learn  more  from  a  text-based  lesson.  Salomon  concluded  that 
learners'  perceptions  of  television  as  an  easy  medium  resulted  in  a  passive  response 
toward  learning  from  that  medium  and,  thus,  reduced  achievement.  The  results  of  these 
studies  have  been  assumed  to  apply  to  learning  from  computers  (Dalton,  1986;  Salomon 
&  Gardner,  1986)  and  interactive  video  (Hannafin,  Garhart,  Rieber  &  Phillips,  1985; 
Schaffer  &  Hannafin,  1986);  however,  there  is  very  little  data  that  has  explicitly 
examined  learners*  preconceptions  of  these  newer  media. 

Although  several  versions  of  schema  theory  have  been  proposed  to  explain  the 
relationship  between  learners'  preconceptions  of  the  ease  of  learning  from  a  medium  and 
their  achievement,  the  construct  of  scripts  proposed  by  Shank  and  Ableson  (1977)  may 
provide  a  reasonable  explanation  for  this  phenomena.  As  described  by  Schank  and 
Ableson  (1977)  scripts  guide  information  processing  by  prescribing  appropriate 
sequences  of  actions  in  well  known  situations.  The  extent  to  which  learners  perceive  a 
medium  as  easy  or  difficult  may  provide  insight  as  to  the  nature  of  the  scripts  that  are 
operating  when  attending  to  a  lesson  presented  through  that  medium. 

Although  earlier  research  (Salomon, 1983;  Salomon,  1984;  Salomon  &  Leigh, 
1984;  Krendl,  1986)  found  that  learners'  preconceptions  of  the  ease  or  difficulty  of 
learning  a  lesson  depended  on  the  medium  of  presentation,  Beentjes  (1989)  found  that 
students'  ratings  of  the  ease  or  difficulty  of  learning  from  a  particular  medium  was 
strongly  dependant  on  the  topic  of  the  lesson. 

It  is  possible  learners  are  aware  that  certain  media  are  more  effective  than  other 
media  in  facilitating  the  acquisition  of  certain  learning  outpomes.  Gagne  (1985) 
classified  objectives  as  psychomotor  skills,  intellectual  skills,  verbal  information,  or 
affective  learning,  depending  on  the  requirements  of  the  task.  Like  many  others,  he 
proposed  that  the  most  effective  medium  for  presentation  is  partially  dependant  on  the 
desired  learning  outcome. 

If  students  are  somehow  aware  that  one  medium  is  more  effective  than  others  for 
the  teaching  of  a  given  outcome,  the  differences  in  perceived  difficulty  observed  by 
Beentjes  (1989)  may  have  been  the  result  of  the  differences  In  learning  domains 
represented  by  the  topics  on  his  questionnaire. 

In  Cennamo's  (In  press)  investigation  of  learners'  preconceptions  of  the  ease  of 
learning  tasks  representing  various  learning  domains  through  the  media  of  books, 
computers,  television,  and  interactive  video,  she  found  that  the  symbol  systems  and 
processing  capabilities  represented  by  the  media  may  have  Influenced  learners' 
perceptions  of  the  ease  of  the  task.  The  students'  ratings  of  the  ease  of  learning  various 
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tasks  using  interactive  video,  computers,  television,  and  books  seemed  to  reflect  some 
awareness  of  the  external  conditions  needed  for  optimal  learning  in  each  domain. 
Learners  rated  interactive  video  and  television  as  the  easiest  from  which  to  learn 
psychomotor  skills.  The  learners1  ratings  of  the  ease  of  learning  psychomotor  skills 
from  television  and  interactive  video  may  have  reflected  an  awareness  that 
demonstrations  are  a  valuable  asset  when  learning  psychomotor  skills.  Likewise, 
modelling  is  suggested  as  a  condition  which  will  facilitate  the  adoption  of  an  attitude 
change,  and  television  and  interactive  video,  the  two  media  where  behavior  can  be  most 
easily  modeled,  were  rated  as  the  easiest  from  which  to  learn  attitude  skills.  On  the 
other  hand,  practice  and  feedback  are  valuable  conditions  for  the  attainment  of  verbal 
information  and  intellectual  skills,  and  the  students  rated  computers  and  interactive 
video,  the  media  which  would  most  easily  provide  practice  and  feedback,  as  the  easiest 
from  which  to  learn  these  skills. 

But  what  influences  learners'  preconceptions  of  media?  Why  do  learners  view 
one  media  as  easier  or  more  difficult  than  another  for  a  particular  task?  This 
exploratory  study  attempted  to  address  these  questions. 

Purpose  of  study.  This  study  was  conducted  as  an  initial  step  in  determining  what 
factors  which  contribute  to  learners'  preconceptions  of  the  ease  or  difficulty  of  learning 
a  specific  task  using  a  specific  medium.  It  examined  whether  the  preconceptions  of  pre- 
service  teachers  varied  by  learning  domain  (verbal,  intellectual,  affective, 
psychomotor)  or  medium  of  presentation  (interactive  video,  computers,  television, 
books).  Information  was  collected  as  to  why  the  students  perceived  one  medium  as  easier 
or  more  difficult  than  another  for  a  particular  learning  task  in  order  to  gain  insight  as 
to  the  factors  that  contributed  to  students'  preconceptions  of  media. 

Methods 

In  this  exploratory  study,  undergraduate  students  were  asked  to  complete  a  self- 
report  questionnaire  on  their  preconceptions  of  the  difficulty  of  four  media  (books, 
television,  computers,  and  interactive  video)  and  to  participate  in  an  individual 
interview. 

Participants 

Forty-two  undergraduate  students  enrolled  in  an  education  class  in  a  large 
midwestern  university  participated  in  the  study.  All  of  the  respondents  were  female. 
Approximately  75%  were  elementary  education  majors  and  25%  were  interested  in 
child  and  family  services.  Twelve  respondents  were  randomly  selected  to  participate  in 
a  follow-up  interview  to  solicit  information  on  factors  that  contributed  to  their 
perceptions  of  ease  or  difficulty. 

Materials 

The  preconceptions  questionnaire  included  a  cover  sheet  that  operationally 
defined  each  medium  in  order  to  ensure  that  all  participants  interpreted  the  terms 
"books"  "television",  "computers",  and  "interactive  video"  in  the  same  manner. 
Information  was  collected  on  the  frequency  of  use  of  each  medium.  The  students  were 
asked  to  rate  the  difficulty  of  learning  12  tasks  using  each  of  the  four  media  on  a  five- 
point  Likert  scale.  One  represented  "very  easy"  and  five  represented  "very  difficult". 
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The  questions  were  similar  in  format  to  those  used  by  Beentjes  (1989)  and 
Salomon  (1984).  Whereas  Beentjes  and  Salomon  ask  the  learners  to  respond  to 
questions  such  as: 

-  How  difficult  would  It  be  for  you  to  learn  to  solve  a  math  problem  from  a  book? 

-  How  difficult  would  it  be  for  you  to  learn  to  solve  a  math  problem  from  television? 
This  questionnaire  included  questions  such  as: 

-  How  ease  would  il  be  to  learn  to  open  and  maintain  a  checking  account  from 


a  book 

very  easy  1 

2 

3 

4 

5 

very 

difficult 

television 

very  easy  1 

2 

3 

4 

5 

very 

difficult 

a  computer 

very  easy  1 

2 

3 

4 

5 

very 

difficult 

interactive  video 

very  easy  1 

2 

3 

4 

5 

very 

difficult 

There  were  three  questions  for  each  learning  domain  (intellectual,  verbal, 
psychomotor,  and  attitudes).  The  outcomes  for  each  learning  domain  were  selected  from 
Dick  and  Carey  (1990)  and  were  assumed  to  fairly  represent  the  type  of  task  that  would 
be  classified  within  a  particular  learning  domain.  The  questions  included  stems  such  as: 

-  How  easy  would  it  be  to  learn  to  change  the  tire  on  an  automobile  from,.. 

(psychomotor) 

-  How  easy  would  it  be  to  learn  to  choose  to  treat  customers  in  a  friendly,  courteous 
manner  from   (attitude) 

-  How  easy  would  it  be  to  learn  to  label  the  parts  of  the  human  body  using  common 
terminology  from  ...  (verbal  information) 

-  How  easy  would  it  be  to  learn  to  open  and  maintain  a  checking  account  from... 

(intellectual  skill) 

Ratings  were  summed  by  medium  and  domain.  Chronbach's  alpha  yielded  an 
acceptable  overall  reliability  coefficient  of  .87.  Alpha  coefficients  for  each  media 
subscale  are  as  follows:  books,  ,83;  television,  .80;  computers,  .63;  interactive  video, 
.77.  Alpha  coefficients  for  each  domain  subscale  were  .72  for  psychomotor,  .83  for 
attitudes,  .60  for  intellectual  skills,  and  .78  for  verbal  information. 

Procedures: 

During  the  seventh  week  of  class,  the  participants  completed  a  questionnaire 
assessing  their  preconceptions  of  the  difficulty  of  each  medium  for  each  domain,  The 
questionnaire  was  presented  as  a  normal  part  of  the  class,  and  all  students  who  were 
present  completed  the  questionnaire.  At  this  time,  the  researcher  invited  the 
participants  to  sign-up  for  an  interview.  Although  the  participants  were  offered  a  small 
amount  of  extra  credit  for  participation  In  tho  intorview,  the  structure  of  the  course 
allowed  the  students  to  earn  course  cmdii  in  a  variety  of  ways  and  they  were  offered  the 
opportunity  to  participate  in  a  varloty  of  ntudios  for  credit  throughout  the  semester. 
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Twelve  students  were  randomly  selected  from  the  26  students  who  volunteered  to 
participate  in  the  interview. 

The  students  who  were  selected  for  individual  interviews  were  asked  to  sign-up 
for  an  interview  appointment.  As  they  reported  to  the  interview  site,  the  interviewer 
explained  the  purpose  of  the  interview  and  presented  the  participants  with  their 
previously  completed  questionnaires.  For  each  task  and  medium,  the  interviewer  read 
the  question  and  the  participants'  responses  aloud.  The  students  were  given  the 
opportunity  to  change  their  responses,  then  asked  to  orally  reflect  on  why  they  rated  the 
medium  as  they  did.  These  conversations  were  audiotaped  and  later  transcribed. 

Results 

The  data  from  the  questionnaires  were  analyzed  using  a  Repeated  Measures 
Analysis  of  Variance  to  determine  whether  students  preconceptions  of  the  ease  of 
learning  a  task  varied  by  learning  domain,  by  medium,  or  randomly  depending  on  topic. 
The  data  from  the  interviews  were  entered  into  a  database  and  sorted  by  question.  The 
responses  for  each  question  were  analyzed  to  determine  trends  in  responses.  The  data 
analysis  indicated  several  significant  findings  concerning  learners'  preconceptions  of  the 
ease  of  learning  a  given  outcome  using  a  given  medium  of  presentation. 

Analysis  of  Questionnaire  Data 

As  expected,  there  was  a  significant  difference  among  media,  F  (1»41)  -18.6, 
p<.001,  and  among  learning  domains,  F(1,41)  =  9.41,  p=.004.  Interactive  video  was 
perceived  to  be  the  easiest  to  learn  from  (M  =  23.14),  and  books  were  perceived  to  be 
hardest  (M  =  31.45).  Computers  (M  =  26.93)  were  perceived  to  be  easier  than  books 
and  television,  and  television  (M  =  28.14)  was  perceived  to  be  easier  than  books. 
Students  perceived  that  it  was  easier  to  learn  verbal  information  (M  *  24.93)  and 
attitudes  (M  =  26.50)  than  intellectual  (M  =  28.50)  and  psychomotor  (M.  *  29.74) 
skills. 

Table  1.  Means  for  media  and  domains 


Books      Television   Computers         IVD     Domain  Total 


Psychomotor 

9.64 

6.05 

8.24 

5.81 

29.74 

Attitude 

7.79 

5.45 

7.52 

5.74 

26.50 

Intellectual 

7.19 

8.69 

6.36 

6.26 

28.50 

Verbal 

6.83 

7.95 

4.81 

5.33 

24.93 

Media  Total 

31  .45 

28.14 

26.93 

23.14 

Note:  n  =  42 


235 


Preconceptions  of  Media 
Page  6 

As  expected,  there  was  also  a  significant  interaction  among  learning  domain  and 
medium,  F(1,  41)  =  35.54,  p<  .001.    Learners  perceived  that  it  was  easier  to  learn 
psychomotor  skills  from  television  (M  «  6.05)  and  interactive  video  (M  -  5.81)  than 
from  computers  (M  =  8.24)  and  books  (M  »  9.64).  For  the  learning  of  attitudes, 
learners  also  perceived  it  to  be  easier  to  learn  from  television  (M  «  5.45)  and 
interactive  video  (M.  »  5.74)  than  from  computers  (M  *  7.52)  and  books  (M  =  7.79). 
Learners  perceived  it  to  be  easier  to  learn  intellectual  skills  from  interactive  video  (M 

6.26)  and  computers  (M  =  6.36)  than  from  books  (M  -  7,19)  and  television  (M  » 
8.69).  The  same  trend  continued  for  learning  verbal  Information:  Computers  (M  » 
4.81)  and  Interactive  video  (M  =  5.33)  were  rated  easier  than  books  (M,  »  6.83)  and 
television       «  7.95). 

In  examining  the  means  by  medium,  ihe  results  Indicate  that  it  Is  porcoivod  to  bo 
easiest  to  learn  verbal  information  from  books  (M  «  6.83),  followed  by  Intellectual 
skills  (M  =  7.19),  attitudes  (M  -  7.79),  and  psychomotor  skills  (M  -  9,64),  For 
television,  it  Is  perceived  to  be  easiest  to  learn  attitudes  (M  *  5.45),  followed  by 
psychomotor  skills  (M  =  6.05),  verbal  information  (M.  «  7.95),  and  Intellectual  skills 
(M  «  8.69).  It  is  perceived  to  be  easiest  to  learn  verbal  information  from  computers 
(M  -  4.81),  followed  by  intellectual  skills  (M.  -  6.36),  attitudes  (M  -7.53),  and 
psychomotor  skills  (M.  =  8.24).  For  interactive  video,  the  learners  perceive  it  to  be 
easier  to  learn  verbal  information  (M  =  5.33),  followed  by  attitudes  (M  =  5.74), 
psychomotor  skills  (M  =  5.81),  then  intellectual  skills  (M  =  6.26). 

Analysis  of  Interview  Data 

The  frequently  listed  responses  were  sorted  into  several  categories.  The 
majority  ot  the  responses  referred  to  the  symbol  systems  offered  by  the  various  media. 
Participants  mentioned  the  presence  or  absence  of  pictures,  graphics,  live 
demonstrations,  moving  images,  and  related  symbols.  Tne  second  most  frequently 
mentioned  category  of  responses  referred  to  the  processing  capabilities  offered  by  the 
media.  Responses  in  this  category  referred  to  the  ability  of  the  media  to  present 
questions,  ease  of  reviewing  Information,  option  of  controlling  the  pace  of  the 
presentation,  possibility  for  practice  with  feedback,  and  similar  responses.  The  third 
most  frequently  mentioned  category  of  responses  involved  the  learners'  preferences  for 
certain  media.  Responses  in  this  category  included  the  learners'  familiarity  with 
learning  a  particular  task  with  a  particular  medium,  the  extent  to  which  they  found 
presentations  using  a  certain  media  interesting,  and  other  personal  preferences.  Other 
responses  mentioned  the  physical  limitations  of  the  technology.  Participants  mentioned 
the  portable  nature  of  books,  the  fact  that  videotapes  could  be  shown  at  certain  locations, 
and  the  accessibility  of  computers  and  books.  A  few  participants  referred  to 
characteristics  of  the  task  when  asked  why  they  rated  a  particular  medium  they  way  they 
did  for  a  particular  task.  These  responses  referred  to  the  perceived  ease  or  difficulty  of 
the  task  in  question. 

Based  on  the  results  of  the  statistical  analysis,  the  interview  data  was 
interpreted  in  terms  of  the  media  by  domain  interaction.  Only  those  responses  that  were 
mentioned  by  five  or  more  participants  will  be  included  in  the  following  discussion. 
Some  of  the  participants  responded  that  computers  or  interactive  video  were  "like 
books"  or  "like  television",  but  provided  no  other  reasons  for  their  responses.  Due  to 
the  lack  of  relevant  Information  in  these  responses,  they  will  not  be  included  in  the 
analysis.  (See  Table  2) 
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Table  2:    Table  of  responses  mentioned  by  five  or  more  participants 


Attitude  Skills 


Books 

self-paced,  can  skim,  can  review  9 

Boring  8 

easy  task  6 

Television 

see  how  to  do,  demo  1  1 

see  it  modeled  8 

interesting,  entertaining  5 

Computers 

same  as  books  1  2 

graphics  and  pictures  5 

read  and  follow  steps  5 

interesting  and  gains  attention  5 

no  live  demos  5 

Interactive  Video 

same  as  TV  12 

demonstrations,  person  models  1  1 
questions,  participate,  interactive  7 

visuals  5 
involvement,  questions,  exercises  6 

TV  and  computer  5 

demonstrate  steps  5 

like  TV  5 


Verbal  information 


Books 

labels,  diagrams,  pictures 
facilitate  memorization 
familiar 


1  2 
8 
6 


Television 

show  instead  of  tell,  point  out  things11 
pace 

visualize/  see  it 
need  interaction  &  hands  on 


Computers 

show  pictures  with  labels 
interaction  &  quizzes 
like  books 
hands-on 

Interactive  Video 

Quiz/  interaction 


1  0 
5 
5 


1  1 
1  0 
9 

9 


Intellectual  skills 


Books 

task  easy  (or  hard)  1  2 

familiar  1 2 

telling  what  to  do,  directions  5 

Television 

lack  of  interaction,  just  watch  1  0 

set  pace  &  can't  review  8 

someone  just  tells  you  6 

Computers 

Like  books  1  2 
practice  problems  and  questions  7 

step-by-step  7 

hands-on  involvement  5 

Interactive  Video 

watch  someone  7 


Psychomotor  Skills 


Books 

Lack  of  demo,  can't  see  it  done  1  1 

step-by  step  Instructions  7 

Television 

see  it  done  2  9 

can  model  7 

Computers 

not  real  pictures  8 

graphics  &  animation  6 

like  books  8 

Interactive  Video 

moving  images  and  sound  8 

control  of  pace  &  review  7 

see  someone  doing  it/  demo  7 

like  TV  7 

questions  and  quizzes  S 
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Psychomotor  skills.   When  learning  psychomotor  skills,  the  participants  felt  it 
was  important  to  be  able  to  see  the  skill  demonstrated.  This  characteristic  was  the  most 
frequently  cited  reason  for  rating  television  and  interactive  v5deo  as  easy,  and  most 
frequently  cited  reason  for  rating  books  as  a  difficult  medium  from  which  to  learn 
psychomotor  skills.  In  examining  the  responses  to  the  question  on  psychomotor  skills 
mentioned  by  five  or  more  participants,  the  vast  majority  of  the  responses  referred  to 
the  symbol  systems  offered  by  the  media  in  question.  Interactive  video  was  the  only 
medium  for  which  there  were  frequent  responses  mentioning  any  other  category.  The 
processing  options  available  through  the  medium  of  interactive  video  were  perceived  as  a 
positive  addition  to  the  ability  to  view  actual  demonstrations.  Several  respondents 
mentioned  that  interactive  video  would  allow  them  to  answer  questions,  review  the 
content,  and  control  the  pace  of  the  presentation, 

Verbal  information.  When  learning  verbal  information,  tho  participants  felt  that 
the  processing  capabilities  offered  by  the  media  were  at  least  as  Important  as  the  symbol 
systems.  The  ability  of  computers  and  interactive  video  to  provide  hands-on  interaction 
in  the  way  of  practice  questions  and  feedback  was  mentioned  quite  frequently  as  a  reason 
for  the  perceived  ease  of  these  two  media.  Conversely,  the  lack  of  learner  control  over 
the  pace  of  a  television  program  was  mentioned  most  frequently  as  a  reason  why 
television  was  rated  as  a  difficult  medium  from  which  to  learn  verbal  information.  The 
symbol  systems  offered  by  the  media  were  also  important.  Pictures  with  labels  were 
viewed  as  positive  features  of  computers  and  books,  and  the  fact  that  someone  was  simply 
"telling*1  you  information  was  viewed  as  a  negative  aspect  of  television.  Respondents  also 
mentioned  that  they  were  familiar  with  learning  verbal  information  from  books  and  that 
they  felt  the  format  of  books  facilitated  memorization  of  verbal  information. 

Attitude  Skills.  The  ability  of  a  medium  to  present  a  live  model  demonstrating  the 
desired  behavior  was  viewed  as  the  most  important  characteristic  for  increasing  the  ease 
of  learning  an  attitude.  This  ability  was  seen  as  a  positive  characteristic  of  television 
and  interactive  video  and  the  lack  of  such  ability  was  seen  as  a  negative  characteristic  of 
computers.  The  ability  of  a  medium  to  provide  visuals,  graphics  and  pictures  was  also 
perceived  to  facilitate  the  ease  of  learning  attitudes.  Learner  preferences  also  were 
viewed  as  very  important  in  the  ease  of  learning  attitudes.  The  participants  frequently 
mentioned  that  learning  an  attitude  from  television  and  computers  would  be  interesting 
and  entertaining,  yet  learning  attitude  skills  from  books  would  be  boring.  However,  the 
self-paced  nature  of  books  was  viewed  as  a  positive  characteristic  in  learning  attitudes 
from  books.  Respondents  liked  the  fact  that  they  could  skim  over  sections  or  review 
sections  when  they  desired.  Other  individuals  mentioned  that  the  ease  of  learning  an 
attitude  from  books  was  determined  by  the  ease  or  difficulty  of  the  task  they  were  to 
learn. 

Intellectual  skills.  Processing  capabilities  were  viewed  as  important 
characteristics  for  facilitating  the  ease  of  learning  an  intellectual  skill.  The  ability  of 
computers  and  interactive  video  to  provide  practice  questions  and  hands-on  involvement 
in  the  learning  of  intellectual  skills  were  frequently  mentioned  responses.  Conversely, 
the  lack  of  hands-on  involvement  was  seen  as  a  characteristic  which  made  it  more 
difficult  to  learn  intellectual  skills  from  television.  The  lack  of  learner  control  over  the 
pace  of  television  and  the  lack  of  the  option  of  reviewing  information  were  frequently 
given  as  reasons  that  participants  rated  television  as  a  difficult  medium  from  which  to 
learn  intellectual  skills.  Responses  which  mentioned  the  symbol  systems  offered  by  the 
respective  media  were  the  next  most  frequently  mentioned  category.  The  ability  of 
computers  and  interactive  video  to  demonstrate  the  skill  in  a  step-by-step  manner  was 
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perceived  as  important  in  facilitating  the  ease  of  learning  intellectual  skills.  Although 
the  ability  to  watch  someone  perform  the  skill  was  perceived  as  a  positive  characteristic 
when  coupled  with  the  processing  capabilities  of  Interactive  video,  the  fact  that  someone 
Hjust  tells  you"  how  to  do  the  skill  was  viewed  as  a  negative  characteristic  of  television. 
For  the  medium  of  books,  the  most  frequent  responses  fell  Into  the  categories  of 
characteristics  of  the  task  and  learner  preferences.  The  ease  or  difficulty  of  the  task 
were  frequently  mentioned  as  reasons  for  participants1  ratings  of  the  ease  or  difficulty 
of  learning  from  books.  Just  as  frequently,  participants  mentioned  their  familarity 
with  learning  from  books  as  a  reason  for  their  responses. 

Discussion 

The  purpose  of  this  study  was  to  determine  factors  which  contribute  to  learners' 
preconceptions  of  the  ease  or  difficulty  of  learning  a  specific  task  using  a  specific 
medium.  Participants  completed  a  self-report  questionnaire  assessing  their 
preconceptions  of  the  ease  of  achieving  various  learning  outcomes  (psychomotor, 
affective,  verbal,  intellectual)  using  the  media  of  interactive  video,  computers,  books, 
and  television,  and  participated  in  individual  interviews  in  which  they  were  asked  to 
verbally  reflect  on  their  responses  to  the  questionnaire. 

The  results  of  this  study  indicated  that  the  participants  viewed  interactive  video, 
computers,  television,  and  books  as  very  different  in  their  ease  of  learning.  Contrary  to 
previous  research  (  Krendl,  1986;  Salomon,  1984;  Salomon  &  Leigh,  1984),  the 
participants  did  not  consistently  rate  books  as  the  most  difficult  medium.  Instead,  there 
was  a  strong  interaction  among  the  medium  of  presentation  and  the  learning  domain  of 
the  intended  outcome.  An  analysis  of  the  interview  responses  indicated  that  for  different 
domains  of  learning  outcomes,  learners  use  different  criteria  for  rating  a  medium  as 
easy  or  difficult. 

The  participants  indicated  that  for  learning  attitudes,  the  ability  of  a  medium  to 
present  actual  models  demonstrating  the  desired  behavior  was  an  important  factor  in 
contributing  to  the  ease  of  learning  from  a  medium.  Therefore,  it  is  not  surprising  that 
television  and  interactive  video  are  rated  as  the  easiest  media  from  which  to  learn 
attitude  skills.  Learners'  personal  preferences  also  seem  to  play  a  part  in  the  perceived 
ease  of  learning  an  attitude  from  a  given  medium;  the  participants  indicated  that  it  would 
be  boring  to  learn  an  attitude  from  books,  while  it  would  be  interesting  and  entertaining 
to  learn  the  same  type  of  skill  from  television. 

The  ability  of  a  medium  to  present  an  actual  demonstration  of  the  target  skill  was 
also  an  important  factor  in  the  participants'  ratings  of  the  perceived  ease  of  learning  a 
psychomotor  skill,  thus,  they  rated  interactive  video  and  television  as  the  easiest  media 
from  which  to  learn  this  type  of  skill.  In  addition,  the  participants  perceived  that  the 
added  processing  capabilities  of  interactive  video  which  allow  them  to  control  the  pace  of 
the  presentation,  respond  to  questions,  and  review  sections  would  further  contribute  to 
the  ease  of  learning  a  psychomotor  skill. 

For  the  learning  of  verbal  information  and  intellectual  skills,  the  participants 
indicated  that  the  processing  capabilities  of  a  medium  would  greatly  contribute  to  the 
ease  of  learning  from  that  medium.  The  ability  of  computers  and  interactive  video  to 
provide  the  learners  with  hands-on  practice  of  the  target  skills  was  viewed  as  an 
important  factor  in  their  ratings  of  these  media.  In  addition,  the  symbol  systems  offered 
by  the  media  were  mentioned  as  influential  factors  in  their  ratings.  The  presence  of 
pictures  with  labels  was  mentioned  frequently  as  an  important  factor  in  learning  verbal 
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information  and  the  step-by-step  demonstration  of  an  intellectual  skill  was  mentioned 
frequently  as  an  influential  factor  which  contributed  to  their  perceptions  of  the  ease  of 
learning  an  intellectual  skill. 

It  is  interesting  to  note  that  learner  preferences  and  task  characteristics  were 
frequently  cited  as  reasons  for  the  participants'  ratings  of  the  ease  of  learning  from 
books.  They  frequently  mentioned  that  they  were  familiar  with  learning  a  particular 
skill  from  books.  In  addition,  the  perceived  ease  or  difficulty  of  the  task  itself  was 
mentioned  frequently  as  a  reason  for  their  ratings.  It  appears  that  the  symbol  systems 
and  processing  capabilities  of  books  are  less  important  factors  in  the  participants' 
ratings  of  this  medium  than  in  their  ratings  of  television,  computers,  and  interactive 
video.  Although  the  participants  may  be  less  familiar  with  computers  and  interactive 
video  than  books,  they  are  assumed  to  be  very  familiar  with  television.  However,  their 
responses  indicate  that  they  may  be  more  familiar  with  receiving  instruction  through 
books  than  through  the  other  three  media. 

Although  the  commonly  accepted  viewpoint  is  that  "...media  are  mere  vehicles 
that  deliver  instruction  but  do  not  influence  student  achievement  any  more  than  the 
truck  that  delivers  our  groceries  causes  chance  in  our  nutrition"  (Clark,  1983,  p. 
445),  Kozma  (1991)  has  challenged  and  expanded  this  commonly  held  view  and 
proposed  that  the  "capabilities  of  a  particular  medium,  in  conjunction  with  methods  that 
take  advantage  of  these  capabilities,  interact  with  and  influence  the  ways  learners 
represent  and  process  information  and  may  result  in  more  or  different  learning  when 
one  medium  is  compared  to  another  for  certain  learners  and  tasks  "  (p.  179).  Kozma 
goes  on  to  say  "Whether  or  not  a  medium's  capabilities  make  a  difference  in  learning 
depends  on  how  they  correspond  to  the  particular  learning  situation-  the  tasks  and 
learners  involved-  and  the  way  the  medium's  capabilities  are  used  by  the  instructional 
design."  (p.  182) 

The  results  of  this  study  suggest  that  learners'  perceptions  of  the  ease  of  learning 
a  particular  task  using  a  particular  medium  are  strongly  influenced  by  the  capabilities 
of  the  medium  such  as  it's  symbol  systems  and  processing  capabilities.  In  addition,  their 
preconceptions  are  also  influenced  by  characteristics  of  the  learner  and  the 
characteristics  of  the  task.  Although  this  study  has  not  attempted  to  determine  which 
medium  actually  facilitated  the  ease  of  learning  a  particular  task,  it  has  attempted  to 
determine  what  factors  contribute  to  learners'  preconceptions  of  the  ease  of  learning  a 
given  task  from  a  given  instructional  delivery  system.  Future  research  should  continue 
to  explore  the  relationship  between  learners'  preconceptions  of  a  medium  and  their 
actual  achievement  when  learning  from  that  medium. 

It  is  important  to  recognize  that  this  exploratory  study  has  been  conducted  with  a 
small  number  of  female  undergraduate  education  students.  It  is  possible  that,  due  to  the 
past  experience  and  academic  major  of  this  group,  this  population  has  an  enhanced 
awareness  of  effective  teaching/learning  strategies  that  may  not  be  present  in  the 
general  adult  population.  Future  research  is  needed  to  determine  to  what  extent  the 
findings  of  this  exploratory  study  apply  to  the  general  adult  population. 
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Abstract 

The  purpose  of  this  study  was  to  investigate  whether  differences  in  the  level  of  visual  complexity 
in  motion  visuals  have  an  effect  on  cognitive  learning  of  students  in  different  grade  levels.  The 
instructional  content  was  a  14-minute  video  lesson  concerning  the  motion  of  objects  in  the  universe. 
A  3  (levels  of  visual  complexity)  x  2  (grade  levels)  factorial,  pretest-posttest  design  was  used.  Three 
4th-  and  three  8th-grade  classrooms  were  randomly  assigned  to  one  of  the  throe  video  treatment 
groups.  All  subjects  received  the  pretest,  treatment,  and  posttest.  Pretest  and  posttest  consisted  of 
28  multiple-choice  questions  and  were  designed  to  test  subject's  comprehension  of  the  instructional 
content.  The  results  indicated  that  providing  different  levels  of  visual  complexity  in  motion  visuals 
had  no  significant  impact  on  children's  comprehension  of  the  intended  learning.  Also,  no  interaction 
between  amount  of  visual  complexity  in  motion  visuals  and  grade  level  was  found. 


Introduction 

With  the  continually  increasing  impact  of  visual  messages  in  our  society,  visual  instruction 
has  become  one  of  the  major  methods  of  communicating  information  in  school  settings.  A  variety 
of  visual  materials,  such  as  drawings,  photographs,  film,  television,  and  computer  software,  is 
commonly  used  to  complement  regular  classroom  instruction  at  all  levels  of  education  (Dwyer, 
1978,  1982-83).  Prior  research  has  demonstrated  that  pictures  are  a  versatile  instructional  tool.  It 
can  help  children  learn  many  different  tasks,  such  as  learning  from  oral  prose  (Levin  &  Lesgold, 
1978),  learning  of  nonfictional  passages  taken  from  newspaper  (Levin  &  Berry,  1980),  learning  of 
written  text  (Levie  &  Lentz,  1982),  and  word  recognition  (Willows,  1978).  However,  studies  of 
visual  instruction  also  show  that  the  inclusion  of  a  picture  in  instruction  does  not  automatically 
improve  student  achievement  (Dwyer,  1978;  Levie  &  Lentz,  1982;  Peeck,  1987).  Different  visuals 
containing  different  amount  of  visual  complexity  may  affect  learning  differently. 

Over  the  years,  two  opposite  approaches  to  designing  instructional  visual  materials  have  arisen 
from  the  concept  of  visual  complexity.  They  are  realism  theory  (Carpenter,  1953;  Dale,  1946; 
Morris,  1946)  and  relevant  cue  hypotheses  (Miller,  1957;  Travers,  1964;  Traversfc  Alvarado,  1970). 
The  basic  assumption  of  realism  theory  is  that  learning  will  be  more  complete  as  the  amount  of 
visual  detail  in  the  instructional  materials  increases.  However,  relevant  cue  hypotheses  contend  that 
complex  visuals  may  be  interesting  to  view,  hut  due  to  the  limited  capacity  of  human  information 
processing,  this  type  of  visuals  is  more  difficult  for  the  brain  to  process. 
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Visual  complexity  can  vary  among  a  great  number  of  different  attributes.  However,  the  amount 
of  interior  detail  in  figures  and  presence  of  background  information  are  two  attributes  which  have 
been  implicated  by  researchers  as  important  factors  influencing  learner's  processing  of  visual  in- 
formation (I)wyer,  1978;  Fleming,  1987;  .Franken,  1977;  Nelson,  Metzler,  &  Reed,  1974;  Travcrs, 
1969). 

In  a  visual  display,  critical  figures  (objects  or  people)  may  show  extreme  variations  in  amount 
of  interior  detail.  The  range  is  from  full  color  photographic  representations  to  black-and-white 
outline  drawings  that  portray  only  the  few  edges  the  artist  considers  to  be  of  crucial  significance 
in  identifying  the  characters.  In  some  studies,  researchers  have  directly  manipulated  the  amount 
of  figure  detail  in  pictures  to  determine  whether  extra  interior  detail  facilitates  different  levels  of 
learning  in  the  cognitive  domain.  For  example,  since  1967  Dwyer  (1978, 1982-83)  and  his  associates 
have  conducted  more  than  100  studies  in  this  area.  The  results  of  these  studies  generally  concluded 
that  employing  highly  detailed,  complex  illustrations  in  visual  lessons  is  unnecessary,  and  it  may 
even  impair  the  intended  learning. 

In  addition  to  Dwyer's  studies,  a  number  of  researchers  have  focused  on  the  roles  that  varying 
levels  of  figure  detail  play  in  picture  memory  (Cody  &  Madigan,  1982;  Evertson  &  Wicker,  1974; 
Jesky  &  Berry,  1991;  Nelson,  Metzler,  &  Reed,  1974;  Ritchey,  1982;  Travers,  1969).  Unfortunately, 
contradictory  results  are  found  among  these  studies.  For  example,  Evertson  and  Wicker  reported 
that  the  retention  of  visual  stimuli  was  positively  related  to  the  amount  of  detail  in  stimuli,  whereas 
Ritchey  found  that  outlines  were  recalled  significantly  better  than  detailed  drawings.  Therefore, 
further  research  is  still  needed  to  determine  the  influence  of  the  addition  of  figure  detail  on  cognitive 
learning. 

The  influence  of  background  information  on  cognitive  learning  is  another  question  examined 
in  the  research  on  visual  complexity.  Many  researchers  have  indicated  that  perception  of  a  visual 
figure  is  determined  not  only  by  its  characteristics  but  also  by  its  surrounding  context  (e.g.,  Antes 
&  Metzger,  1980;  Fleming,  1987).  In  general,  background  information  refers  to  the  excess  sur- 
rounding information  in  which  the  critical  figure  is  embedded  (Borg  &  Schuller,  1979;  Fleming  & 
Levie,  1978;  Sparks,  1973).  According  to  this  definition,  pictures  may  vary  in  the  extent  to  which 
the  critical  figure  in  the  picture  is  presented  in  context  or  in  isolation.  A  number  of  empirical  in- 
vestigations have  focused  on  the  presence  or  absence  of  background  information  in  a  visual.  One  of 
the  earliest  reported  studies  was  conducted  by  Spaulding  (1956).  He  found  that  visual  information 
unnecessary  to  critical  figures  should  be  eliminated  because  it  may  motivate  an  interpretation  that 
is  not  compatible  with  the  purpose  of  the  illustration.  However,  in  contrast  to  Spaulding's  find- 
ings, Antes  and  Metzger  (1980)  reported  that  the  context  (background)  of  objects  helped  learners 
construct  a  general  characterization  of  the  pictures  which  provided  learner  expectancies  to  perform 
their  discrimination  task.  Therefore,  the  research  results  concerning  the  influence  of  background 
information  on  cognitive  learning  thus  far  are  inconclusive  and  usually  contradict  one  another. 

While  the  research  indicates  that  visuals  containing  varying  levels  of  complexity  have  different 
effects  upon  learning,  many  researchers  find  that  these  effects  interact  with  learner  age.  A  number 
of  empirical  studies  have  been  conducted  which  attempt  to  establish  the  relationship  between  the 
c  omplexity  of  a  picture  and  a  child's  ability  to  process  it  (e.g.,  Collins,  1970;  Evertson  &  Wicker, 
107-1;  Hagen,  1972;  Hale  k  Taweel,  1974;  Moore  &  Sasse,  1971;  Pezdek  &  Chen,  1982;  Pezdek, 
1987).  Although  the  research  results  appear  inconsistent  to  some  extent,  in  general,  researchers 
agree  that  there  are  age  differences  in  how  the  children  allocate  their  attention  to  a  visual  task. 
Young  children  tend  to  center  perceptually  on  selected  aspects  of  a  picture,  while  older  students 
are  better  able  to  view  the  picture  in  a  more  global  manner.  Unfortunately,  however,  at  present 
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there  is  still  very  little  guidance  for  educators  in  selecting  visuals  that  are  appropriate  for  use  with 
specific  groups  of  learners. 

In  reviewing  the  studies  concerning  the  effect  of  visual  complexity  on  learning,  it  is  evident  that 
they  frequently  have  used  static  visual  stimuli  as  the  material  to  bo  learned.  Only  a  few  studies 
have  attempted  to  investigate  the  relative  effectiveness  of  motion  visuals  that  employ  different  levels 
of  visual  complexity  to  complement  verbal  instruction  (Acker  &  Klein,  1986;  Chou,  1991;  Dwyer, 
1970;  Hozaki,  1988).  In  view  of  the  increasing  use  of  motion  visuals  in  school  settings  today,  there 
is  no  doubt  that  more  research  work  needs  to  be  conducted  regarding  this  area. 

The  purpose  of  this  study  was  to  examine  whether,  or  to  what  extent,  differences  in  the  level  of 
visual  complexity  in  motion  visuals  have  an  effect  on  cognitive  learning  of  students  in  different  grade 
levels.  Specifically,  the  amount  of  figure  detail  and  background  information  were  two  attributes  of 
visual  complexity  examined  in  this  study. 

Methods 

Subjects 

Subjects  for  the  study  consisted  of  three  -1th  grade  classes  (N  =  78)  and  three  8th-grade  classes 
(N=72)  from  two  public  schools  in  north  central  Florida.  They  were  randomly  chosen  by  the  two 
school  principals  and  the  school  curriculum  coordinators.  All  of  the  classes  selected  were  average 
classes  and  were  similar  to  one  another. 

Instructional  Content 

The  instructional  content  selected  for  thin  study  was  a  14-minute  video  lesson  concerning  the 
motion  of  objects  in  the  universe.  The  lesson  was  divided  into  three  units:  (a)  motion  and  force, 
(b)  gravity,  and  (c)  the  law  of  inertia.  In  each  unit ,  a  related  question  was  first  posed,  and  then  the 
basic  concept  as  well  as  some  real-life  examples  of  this  concept  were  explained  and  demonstrated. 
During  the  development  of  the  lesson  content,  it  was  reviewed  periodically  by  practicing  teachers  of 
4th  and  8th  grades  so  that  both  grade  students  could  easily  understand  the  instructional  material. 

Instructional  Treatments 

The  treatments  for  this  study  were  produced  in  two  phases.  In  phase  one,  three  versions 
of  black-and-white  computer  animation  were  created  using  the  ADDmotion  animation  program 
that  runs  on  a  Macintosh  environment.  After  the  three  versions  of  computer  animation  had  been 
produced,  the  next  phase  was  to  convert  them  to  be  three  VHS  videotapes  by  recording  the  image 
from  a  computer  monitor  with  a  professional  studio  camera. 

The  videotapes  represented  three  different  levels  of  visual  complexity  in  motion  visuals  for 
this  study:  outline  drawings  (01)),  outline  drawings  with  added  figure  detail  (ODF),  and  outline 
drawings  with  added  figure  detail  and  background  information  (ODFB).  Although  the  three  versions 
of  the  video  programs  had  different  levels  of  visual  complexity,  they  all  employed  the  identical  verbal 
narration  to  complement  the  visual  information. 

In  the  OD  version,  the  computer  animation  contained  only  the  outline  shapes  of  critical  objects 
necessary  to  convey  the  intentional  meaning  of  the  picture.  Peripheral  information,  such  as  figure 
detail  and  background,  was  not  included  (see  Figure  1). 

In  the  ODF  version  of  computer  animation,  interior  detail  in  figures  was  added  to  each  frame  of 
the  OD  version.  If  the  critical  information  of  a  frame  was  a  girl  pushing  a  toy  car,  for  example,  the 
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Figure  1:  A  Sample  Frame  in  Outline  Drawings  (OD) 


added  interior  detail  would  include  the  designs  on  the  girl's  clothing  as  well  a*  shade  and  design 
patterns  in  the  toy  car  (see  Figure  2). 

The  ODFB  version  was  presented  with  the  same  visual  information  as  the  ODF  version  except 
that  appropriate  background  information  was  provided  in  each  frame.  For  example,  the  visual  of  a 
girl  pulling  a  toy  car  in  the  ODF  version  would  be  embedded  in  a  living  room  in  the  ODFB  version 
(see  Figure  3). 

In  order  to  eliminate  variables  which  may  confound  the  research  results,  all  pictures  were  de- 
veloped as  black-and-white  representations.  In  this  way,  the  possibility  of  color  effect  could  be 
avoided. 

Instrumentation 

The  pretest  measure  (K-R  20  =  .87)  dealt  with  the  specific  content  covered  in  the  selected 
video  lesson.  It  consisted  of  28  multiple-choice  questions  which  tested  students'  comprehension  of 
content  which  would  be  presented  in  the  video  lesson.  The  posttest  was  identical  in  style  to  the 
pretest  and  consisted  of  the  same  28  items  but  arranged  in  a  different  order.  The  posttest  was 
administered  to  the  subjects  two  weeks  after  the  pretest  was  given. 

These  measures  required  that  the  student  had  a  thorough  understanding  of  the  concept  of 
motion,  gravity,  and  the  law  of  inertia.  They  were  designed  to  determine  whether  the  students 
comprehended  what  was  communicated  and  could  use  the  information  being  received  to  explain 
some  other  phenomena. 

Procedure 

Three  4th-  and  three  8th-grade  classrooms  were  randomly  assigned  to  one  of  the  three  video 
treatment  groups-OD,  ODF,  and  ODFB  group.  After  being  assigned  the  treatment  groups,  the 
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subjects  received  the  pretest  to  determine  their  prior  knowledge  level  in  the  instructional  content. 
However,  since  the  possibility  existed  that  the  pretest  might  activate  students'  attention  toward 
some  specific  areas  of  the  video  program  when  they  would  watch  it  later,  the  pretest  was  adminis- 
tered two  weeks  before  the  experimental  treatments  were  delivered. 

During  the  two- week  period  between  the  pretest  and  the  experimental  treatment,  the  teachers 
were  asked  not  to  teach  the  content  areas  concerning  motion  and  force,  gravity,  and  the  law  of 
inertia.  They  were  also  requested  not  to  answer  students  questions  relating  to  these  areas. 

After  two  weeks,  subjects  received  their  respective  experimental  treatments.  The  video  treat- 
ment was  played  on  a  20-inch  monitor.  Upon  the  completion  of  playing  the  experimental  treatment, 
subjects  received  the  posttest. 

Results 

The  design  of  the  experiment  was  a  3  (levels  of  visual  complexity)  X  2  (grade  levels)  factorial, 
protost-posttcst  design.  In  order  to  determine  whether  there  were  existing  group  differences  in 
achievement  levels  of  subjects  in  different  treatments,  a  pretest-posttest  correlation  and  an  ANOVA 
procedure  were  utilized.  Pearson  correlation  coefficients  indicated  that  pretest  was  highly  correlated 
to  posttest  (r=.7f>)  and  an  analysis  of  variance  also  showed  a  significant  difference  (F=5.94,  p< 
.005)  for  pretest  scores  among  the  three  treatment  groups.  Therefore,  an  analysis  of  covariance 
procedure  ( ANCOVA)  was  used  for  analyzing  main  effects  and  their  interactions  in  this  study,  while 
the  pretest  scores  served  as  a  covariate  to  partial  out  their  effect. 

The  pretest  means,  posttest  means,  standard  deviations,  and  adjusted  means  for  the  three 
treatment  groups  by  grade  levels  are  presented  in  Table  1.  Although  the  trend  of  the  adjusted 
mean  scores  for  the  three  treatment  groups  in  4th  and  8th  grade  was  different,  the  scores  differed 
very  little  from  group  to  group,  a  fact  which  was  reflected  in  the  small  F  ratios  of  the  analysis  of 
covariance  procedure. 


Table  1 

Pretest  Mean,  Posttest  Mean,  Adjusted  Mean  by  Treatment  Condition 


4th  Grade 

8th  Grade 

Overall 

Pre  M 

Post  M 

Adj  M 

Pre  M 

Post  M 

Adj  M 

Adj  M 

(SD) 

(SD) 

(SD) 

(SD) 

Ob 

12.79 

18.04 

17.90 

21.08 

23.04 

22.14 

19.83 

(5.46) 

(7.38) 

(4.68) 

(5.45) 

ODF 

14.67 

18.67 

16.81 

20.00 

22.67 

22.47 

19.65 

(4.10) 

(5.58) 

(4.38) 

(4.00) 

ODFH 

10.48 

13.5!) 

15.57 

17.55 

19.90 

21.30 

18.30 

(4.77) 

(6.01) 

(7.04) 

(6.31) 

Analysis  of  covariance  procedures  demonstrate  that  there  were  no  significant  differences  for  the 
main  effect  of  treatment  (F=1.64,  p>  .05),  grade  (F=0.08,  p>  .05),  and  interaction  effect  (F=0.81, 
p>  .05).  In  addition,  the  F  tests  were  also  computed  separately  within  each  grade.  All  of  the 
respective  results  are  shown  in  Table  2,  3,  and  4. 
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Table  2 

Analysis  of  Covariance  Summary 


Source  DF 

SS 

MS  F 

Treatment  2 

61.5ft 

30.79  1.64 

Grade  1 

1.42 

1.42  0.08 

Treatment  x  Grade  '2 

30.$) 

1  r  17     n  W  I 
I «).  I  /      U.rt  i 

'1  able 

Analysis  of  Covariance  for  Grade 

1  (N  =  7H) 

Source  DF 

SS 

MS  F 

Treatment  2 

66.5-1 

30.79  1.57 

Table  4 

Analysis  of  Covariance  for  Grade  8  (N=72) 

Sour-e  DF 

SS 

MS  F 

Treatment  2 

15.54 

7.76  0.50 

Discussion 

The  results  of  this  study  indicate  that  providing  different  levels  of  visual  complexity  in  motion 
visuals  has  no  significant  impact  on  children's  learning.  The  use  of  complex  visuals  contributes 
little  to  facilitate  children's  comprehension  of  the  intended  learning.  These  findings  are  in  contrast 
to  some  earlier  research  which  has  previously  reported  that  learning  would  be  more  complete  as  the 
amount  of  visual  information,  figure  detail  or  background,  in  the  instructional  materials  increased 
(Antes  &  Metzger,  1980;  Fvcrtson  Wicker,  1974;  Snowman  *■  Cunningham,  197G).  Although  the 
present  study  does  not  find  that  the  most  complex  version  of  motion  visuals  (ODFB)  impairs  the 
intended  learning,  it  supports  Dwyer's  ( I97K)  observation  that  employing  highly  detailed,  complex 
illustrations  in  visual  lessons  is  unnecessary, 

There  is  no  interaction  between  amount  of  visual  complexity  in  motion  visuals  and  grade  level. 
The  differences  among  the  adjusted  mean  scores  for  both  <lth-  and  Hth-grader  in  three  treatment 
groups  are  small.  A  number  of  researchers  have  theorized  and  tested  the  hypothesis  that  there 
is  a  developmental  increase  in  ability  to  process  visual  information;  specifically,  young  children 
have  greater  difficulty  in  selectively  focusing  their  attention  on  the  essential  learning  task  than 
adolescent  children  (Collins,  1970;  Hagen  1972;  Hale,&  Taweel,  1974).  However,  this  hypothesis 
is  not  supported  in  the  present  study.  The  adjusted  mean  scores  among  three  treatments  for  4th- 
graders  do  show  variance  (OD=17.90,  ODF=16.81,  ODFH=15.57),  but  are  not  significant.  With 
regard  to  8th-graders  the  variances  of  three  adjusted  mean  scores  are  even  smaller  (OD  =  22.14, 
Ol)F=22.47,ODFB=22.30).  Therefore,  this  study  supports  the  results  of  Gorman  ( 1973)  in  finding 
that  the  amount  of  visual  complexity  in  visual  instruction  has  no  effect  on  cognitive  learning  of 
students  in  different  grade  levels. 


ERLC 


£53  2*° 


8 


Two  possible  explanations  may  have  accounted  for  the  lack  of  significant  differences  in  this  study. 
First,  in  contrast  to  the  prior  research  which  was  generally  limited  to  addressing  the  effects  of  visuals 
complexity  on  recognition  and  recall,  the  present  study  focuses  on  children's  comprehension  of  visual 
information.  Sigel  (1978)  has  indicated  that  comprehension  must  be  distinguished  from  recognition. 
To  comprehend  a  picture  is  to  extract  meaning  from  the  picture,  whereas  to  recognize  refers  to  the 
identification  or  labeling  of  a  picture  and  does  not  require  understanding.  In  the  literature,  only  a 
few  studies  have  attempted  to  include  comprehension  as  one  of  the  learning  tasks  to  be  examined 
(e.g.  Dwyer,  1971;  Hozaki,  1988).  However,  unlike  other  instructional  tasks  such  as  identification 
and  psychomotor  performance  investigated  in  Dwycr's  and  Hozaki's  studies  respectively,  various 
levels  of  visual  complexity  seldom  make  a  significant  impact  on  comprehension.  Thus,  this  implies 
that  humans  may  use  different  cognitive  processes  to  recognize  and  comprehend  a  visual  display. 
However,  it  is  still  not  clear  how  these  two  processes  are  different.  Further  research  has  to  be  done 
before  we  can  draw  a  general  conclusion  on  this  issue. 

A  second  possible  reason  accounting  for  the  lack  of  difference  may  have  been  that  an  insufficient 
amount  of  pictorial  information  was  added  to  the  two  complex  versions  of  treatments.  In  reviewing 
prior  research  in  the  field,  most  studies  employed  a  much  wider  range  of  variations  in  visual  com- 
plexity. For  example,  Chou  (1991)  used  realistic  video  and  animated  simple  computer-generated 
line  graphics  as  tin*  two  levels  of  visual  instruction.  In  Nelson,  Metzler,  and  Reed's  (1974)  study  the 
three  visual  stimulus  conditions  presented  to  subjects  displayed  significant  variance:  photographs, 
embellished  line  drawings,  and  mnembellished  line  drawings.  In  contrast,  the  three  treatments  used 
in  the  present  study  are  all  computer-generated  line  drawings,  though  figure  detail  and  background 
information  have  been  added  to  the  two  complex  versions.  Gorman  (1973)  has  suggested  that  the 
irrelevant  detail  did  not  interfere  with  the  processing  of  the  relevant  information  presented  due  to 
a  tolerance  level  existed  for  some  irrelevant  information  in  a  picture.  Apparently,  this  research 
study  is  supportive  of  Gorman's  observation.  However,  it  is  still  not  clear  what  the  limits  of  the 
tolerance  level  is.  Therefore,  if  the  complexity  among  three  visual  treatments  in  this  study  could  be 
increased  to  include  a  realistic  video  version,  significant  achievement  differences  among  treatments 
may  occur. 

Several  important  implications  for  educational  researchers  and  instructional  designers  are  pro- 
vided by  this  study.  According  to  the  subjects'  progress  in  their  achievement  scores,  it  is  clear  that 
the  use  of  visual  instruction  does  improve  student's  learning.  However,  the  three  levels  of  visual 
complexity  in  motion  visuals  employed  in  this  research  study  do  not  have  any  differential  impact 
on  children's  comprehension  of  the  intended  learning.  The  more  complex  visuals  are  no  more  ef- 
fective than  the  simple  one.  Therefore,  the  issue  of  whether  to  add  extra  pictorial  information 
in  instructional  materials  appears  to  be  an  economic  rather  than  an  instructional  consideration. 
Instructional  designers  should  first  consider  their  budget  limitations  before  making  a  final  decision 
in  selecting  and  producing  appropriate  visuals  for  learners. 

The  results  of  this  study  offers  evidence  that  there  is  no  significant  relationship  between  the 
treatments  and  grade  levels  on  learner's  comprehension.  No  treatment  condition  serves  to  provide 
students  in  different  grade  levels  with  a  more  efficacious  visual  design  strategy  for  affecting  their 
achievement.  Therefore,  visual  complexity  may  not  be  an  important  factor  to  be  considered  when 
teachers  and  instructional  designers  determine  how  to  present  motion  visuals  in  order  to  facilitate 
the  comprehension  of  learners  at  different  ages.  Hither  a  common  simple  outline  drawing  or  detailed 
drawing  with  background  format  can  be  used  with  equal  effectiveness  with  a  wide  range  of  students 
from  grade  4  through  grade  8. 

Another  implication  provided  from  this  study  is  that  since  level  of  visual  complexity  is  not 
a  physical  attribute  restricted  within  a  specific  medium,  the  findings  of  this  study  may  broadly 
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apply  to  learning  from  film,  television,  computer,  interactive  video,  or  multimedia  systems.  That 
is,  extra  pictorial  information  cannot  facilitate  comprehension  of  the  information  presented  in  the 
motion  visuals.  Even  though  advanced  media  such  as  videodisc  or  digital  video  interactive  (I)VI) 
can  present  information  that  is  more  realistic,  the  incorporation  of  complex  and  detailed  visuals 
into  instructional  materials  is  not  necessary. 

Recommendations 

The  results  of  this  empirical  study  support  the  need  for  more  extensive  investigation  concerning 
the  design  of  visual  instruction.  The  following  recommendations  are  made  for  additional  study  in 
this  area: 

1.  Different  cognitive  learning  tasks  could  be  employed  to  determine  if  the  factor  of  visual 
complexity  only  influences  specific  cognitive  learning  tasks. 

2.  A  wider  range  of  variations  in  visual  information  among  treatments  could  be  used  to  determine 
if  the  level  of  complexity  in  motion  visuals  affects  children's  learning. 

3.  Subjects  at  a  greater  age  difference  could  be  used  to  determine  whether  the  effects  of  visual 
complexity  in  motion  visuals  interact  with  learner  age. 

4.  Studies  could  be  designed  to  utilize  different  subject  content  areas  to  determine  if  the  effects 
of  visual  complexity  are  restricted  to  the  specific  subject  matters. 

5.  Color  and  other  cueing  strategies  could  be  added  to  the  motion  visuals  to  investigate  if 
children's  comprehension  is  affected  by  them. 

(i.  Besides  a  multiple-choice  test,  other  types  of  instruments  could  be  included  to  examine  if 
different  results  from  the  present  study  would  be  found. 

7.  Since  a  statistical  method  (ANCOVA)  is  used  in  the  present  study  to  control  subject's  existing 
differences,  the  same  research  could  be  replicated  randomly  assigning  individual  subjects  to 
treatments  to  determine  if  the  class  room  effect  influences  the  results  of  this  study. 
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The  shift  from  the  behaviorist  paradigm  to  the  cognitive  paradigm  has  led  re- 
searchers to  turn  their  attention  to  cognitive  processes.  Although  the  research  has 
been  impressive,  it  sometimes  fails  to  translate  clearly  into  instructional  practice. 
Perhaps  no  place  is  this  more  apparent  than  in  the  study  of  analogies.  Basic  re- 
search has  provided  insight  into  the  nature  and  mechanism  of  analogies  (e.g., 
Gentner,  1983;  Rumelhart  &  Norman,  1981;  Sternberg,  1977a,  1977b),  shown  that 
analogical  reasoning  ability  is  a  poor  predictor  of  "the  ability  to  infer  from  analo- 
gies" (Yacci,  1990),  and  demonstrated  that  learners  can  be  taught  to  generate 
analogies  (e.g.,  Lowenthal  &  Pons,  1987).  Another  line  of  research  has  focused  on 
instructional  applications,  usually  instruction-provided  analogies.  These 
applications  have  typically  been  in  math  and  science  with  lower-level  learning 
outcomes  (e.g.,  Zeitoun,  1984).  Instructional  analogies  have  been  found  beneficial 
for  learners  from  fifth  grade  (Simons,  1984)  through  college  (Newby  &  Stepich, 
1991).  Occasionally  researchers  have  investigated  the  role  of  analogies  in 
facilitating  higher-order  outcomes:  For  example,  Gick  and  Holyoak  (1980,  1983) 
found  that  providing  multiple  analogs  (i.e.,  cases  or  stories  analogous  to  the  target 
problem)  can  facilitate  problem  solving,  although  single  analogs  were  generally 
effective  only  40%  of  the  time  unless  subjects  were  prompted  to  apply  the  analog. 

While  instruction-provided  analogies  require  more  instructional/learning 
time,  performance  gains  generally  offset  the  additional  time  required  (Newby  & 
Stepich,  1991;  Simons,  1984).  Nevertheless,  the  cognitive  paradigm  and  growing 
interest  in  constructivism  suggest  a  different  perspective:  Must  analogies  remain 
primarily  time-consuming  instruction-provided  strategies,  or  can  learners  be  em- 
powered to  spontaneously  generate  analogies  to  facilitate  higher-order  learning, 
and,  if  so,  how? 

Several  lines  of  research  guided  our  investigation  into  these  questions.  First, 
Gick  and  Holyoak  (1980,  1983)  showed  that  reading  two  relevant  story  analogs  im- 
mediately before  being  asked  to  solve  a  problem  facilitates  the  generation  of  a  suc- 
cessful solution.  However,  since  many  of  their  subjects  could  also  generate  the  ap- 
propriate solution  from  a  single-story  analog  after  they  were  prompted  to  use  it  in 
solving  the  problem,  other  cognitive  processes  might  underlie  the  spontaneous  gen- 
eration of  analogies  to  solve  problems.  Could  subjects  be  taught  to  apply  more  effec- 
tive cognitive  strategies  on  single  analogs  rather  than  having  to  study  multiple 
analogs?  The  former  would  potentially  be  a  more  efficient  use  of  time  and  be 
available  for  wider  transfer. 

The  importance  of  a  broad  knowledge  base  is  supported  by  another  line  of  re- 
search (e.g.,  Brown,  Campione,  &  Day,  1981)  which  suggests  that  efforts  to  develop 
spontaneous  analogical  thinking  may  depend  both  on  developing  general  knowl- 
edge ("world  knowledge")  and  domain-independent  strategies.  Instructional 
strategies  to  develop  spontaneous  analogical  reasoning  are  consistent  with  this 
view  since  they  teach  students  to  process  information  deeply  by  engaging  in  mean- 
ingful elaboration. 

In  the  series  of  experiments  by  Gick  and  Holyoak  (1980,  1983)  subjects  read  one 
or  two  story  analogs;  subjects  were  usually  told  that  they  would  be  tested  on  the 
story.  Various  orienting  tasks  were  required,  most  of  which  required  little  cogni- 
tive processing  (e.g.,  single-sentence  oral  summaries  of  the  whole  story).  Before 
they  were  tested  on  the  story,  subjects  were  asked  to  solve  Duncker's  (1945)  cancer- 
radiation  problem  which  was  usually  introduced  as  a  tangential  task.  This  case 
posed  the  problem  of  how  to  irradiate  a  tumor  at  sufficiently  high  doses,  while 
avoiding  damage  to  surrounding  cells.  The  key  solution  to  the  problem  was  to  si- 
multaneously irradiate  the  tumor  at  low  doses  from  different  angles,  resulting  in  a 
high  cumulative  dose  nt  the  tumor  site.  An  analogous  solution  is  suggested  by  The 
Attack-Dispersion  Story  (see  Appendix),  presented  earlier  to  subjects,  wherein 
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armies  avoided  tripping  land  mines  by  dispersing  troops  along  several  different 
roads  into  a  city. 

The  present  study  made  use  of  Gick  and  Holyoak's  content,  and  drew  from 
Wittrock  and  Alesandrini's  (1990)  research  method.  All  subjects  in  our  study  re- 
ceived The  Attack-Dispersion  Story.  Depending  on  their  assigned  treatment,  sub- 
jects were  asked  to  process  the  story  in  different  ways: 


Summarize: 

Develop  an 
analogy  for: 


Each  paragraph  of  the  story 

The  whole  story 

Each  paragraph  of  the  story 

The  whole  story 

A  fifth  group  was  asked  to  simply  record  the  time  of  completion  for  each  paragraph, 
resulting  in  a  2  x  2  factorial  design  with  an  additional  control  group. 

Our  experiment  differed  from  the  Gick  and  Holyoak  studies  in  significant 
ways,  particularly  relating  to  the  generative  model  of  learning  (Wittrock,  1974). 
First,  the  Gick  and  Holyoak  studies  required  little  cognitive  processing  of  the  con- 
tent; our  experiment  compared  the  effect  of  having  to  cognitively  process  each  para- 
graph or  the  whole  story  (by  summarizing  or  by  generating  an  analogy)  on  the 
ability  to  solve  the  radiation  problem.  Second,  the  Gick  and  Holyoak  studies  did 
not  diroct  the  subjects  to  express  the  summaries  totally  in  their  own  words,  a  re- 
quirement which  is  expected  to  facilitate  more  meaningful  encoding  (Wittrock  & 
Alesandrini,  1990).  Since  conceptual  analogies  facilitate  creative  transfer  of 
knowledge,  while  comprehensive  analogies  facilitate  storing  information  (Stepich 
&  Newby,  1988;  Simons  [1984]  calls  these  respectively  "unconnected"  and 
"connected"  analogies),  we  hypothesized  that  having  to  generate  summaries  or 
analogies  for  each  part  of  the  story,  rather  than  just  for  the  whole,  would  make  the 
individual  elements  of  the  story  analog  more  accessible  and  facilitate  creative  ap- 
plication (far  transfer).  This  hypothesis  also  seems  to  be  supported  by  Mayer's 
( 1980)  theory  of  assimilative  encoding:  The  assimilative  encoding  process  results 
in  a  broader  cognitive  structure,  which  supports  transfer  to  creative  problem  situa- 
tions'* (p.  109). 

The  present  study  examined  whether  an  experimental  treatment  such  as  the  one 
used  by  Wittrock  and  Alesandrini  (1990),  which  encourages  generativity  and 
meaningful  encoding,  can  facilitate  spontaneous  analogical  thinking.  Our  study 
differs  from  that  study  in  that  it  is  measuring  spontaneous  generation  of  a  relevant 
analog  in  solving  a  problem  rather  than  the  amount  of  factual  material  learned. 
Because  of  the  limited  access  to  students  for  testing,  we  did  not  examine  relation- 
ships between  verbal  ability,  analytic  ability,  and  ability  to  solve  the  problem  suc- 
cessfully. 

Method 

Hypotheses 

We  hypothesized  that  (a)  generating  summaries  or  analogies  for  an  analog 
story,  described  in  the  procedure  section,  would  facilitate  subjects'  abilities  to  gen- 
erate the  analogical  dispersion  solution  to  the  radiation  problem  and  (b)  generating 
summaries  or  analogies  for  each  part  of  the  analog  story  would  further  facilitate 
subjects'  abilities  to  generate  the  dispersion  solution  to  the  radiation  problem. 
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Subjects 


Seventy-one  students  enrolled  in  five  courses  in  the  graduate  school  of  educa- 
tion at  a  public,  urban  university  participated  voluntarily  in  this  study.  We  fo- 
cused on  graduate  students  because  we  believed  that  they  would  already  know  how 
to  generate  summaries  and  analogies.  Three  subjects  were  eliminated  because 
they  were  already  familiar  with  the  criterion  problem. 


The  Attack-Dispersion  Story  (Gick  and  Holyoak,  1980)  served  as  the  story  ana- 
log (see  Appendix).  The  story  was  divided  into  4  paragraphs,  rather  than  2,  to 
break  the  story  into  more  salient  parts  and  to  allow  for  more  summaries  and 
analogies  according  depending  on  the  treatment.  All  instructions  were  included 
in  the  test  packets.  Subjects  in  the  summary  groups  were  instructed  to  write  a 
summary  (of  each  paragraph  or  of  the  whole  story).  Subjects  in  the  analogy  group 
were  instructed  to  write  an  analogy  (of  each  paragraph  or  the  whole  story)  and  re- 
ceived the  following  example: 

Each  analogy  should  consist  of  one  or  two  sentences  and  should  clearly  re- 
late the  new  ideas  to  familiar  things.  For  example, 


Subjects  were  presented  with  the  story  prior  to  the  Radiation  Problem  (Duncker, 
1945;  see  Appendix),  which  served  as  the  test  problem. 

Procedure 

Subjects  were  told  the  purpose  was  to  "study  how  people  learn  from  written  mate- 
rial. .  .  .You  will  be  asked  to  study  a  very  short  story.  Then  we  will  give  you  a 
written  test  to  find  out  what  you  learned  from  the  passage.  The  information  we  ob- 
tain in  this  way  may  be  of  help  in  improving  students'  learning  strategies."  In 
th  ree  of  the  classes,  the  professors  allowed  class  time  for  those  who  wished  to  partic- 
ipate; in  the  other  two  classes,  the  experiment  was  administered  immediately  after 
or  before  class  (the  students'  choice).  Students  in  each  class  were  randomly  as- 
signed to  five  groups  in  a  2  x  2  design  (4  treatments  plus  a  control  group);  encoding 
task  (generate  summary,  generate  analogy)  and  level  of  processing  (each  para- 
graph, whole  story). 

Separate  booklets  were  prepared  for  each  treatment,  and  included  the  instruc- 
tions, the  analog  story  (The  Attack-Dispersion  Story),  the  radiation  problem,  and  a 
questionnaire  to  collect  demographic  data  (prior  knowledge  of  the  radiation  prob- 
lem, degrees,  majors,  how  many  graduate  credits  they  have  completed,  age,  and 
sex).  Subjects  in  each  treatment  read  The  Attack-Dispersion  Story  and  generated 
summaries  or  analogies;  subjects  in  the  control  group  read  each  paragraph  of  the 
story  twice,  recording  the  time  after  each  reading.  After  they  had  finished  the 
treatment,  the  radiation  problem  was  presented  as  an  extraneous  problem  so  that 
subjects  would  not  be  prompted  to  connect  the  story  to  the  problem.  They  were  in- 
structed to  write  all  the  solutions  they  could  think  of. 

Evaluation.  The  summaries  and  analogies  subjects  generated  during  the 
treatment  were  evaluated  using  a  procedure  derived  from  Wittrock  and 
Alesandrini  (1990)  to  measure  the  extent  to  which  subjects  followed  the  directions 
and  the  extent  to  which  the  treatments  induced  the  intended  effects  upon  generation 
of  the  analog  solution. 

Subjects  in  the  summary  treatments  received  separate  scores  for  following  di- 
rections and  for  including  salient  content.  Subjects  in  the  paragraph-summary 
treatment  received  2  points  per  paragraph  if  they  wrote  a  correct  summary,  1  point  if 
they  relied  on  words  from  the  story,  and  0  points  if  they  wrote  nothing  (8  points  max- 
imum). They  received  1  point  per  paragraph  for  salient  content:  (a)  routes  radiat- 
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ing  from  fortress;  (b)  full  force  required  for  the  attack;  (c)  total  force  impossible 
from  one  route  without  producing  unwanted  harm;  (d)  dividing  into  small  groups, 
using  different  routes,  and  converging  simultaneously  (4  points  maximum). 
Subjects  in  the  whole-summaries  treatment  were  evaluated  similarly,  with  2  points 
maximum  for  following  directions  and  4  points  maximum  for  content. 

Subjects  in  the  analogy  treatments  received  a  single  score  focusing  on  follow- 
ing the  directions  (responses  were  often  so  loosely  worded  that  it  was  impossible  to 
rate  the  correspondence  between  the  story  and  the  experience  the  subject  was  recall- 
ing). Subjects  in  the  paragraph-analogy  treatment  received  3  points  per  paragraph 
for  generating  real  analogies,  2  points  for  generating  loose  analogies,  1  point  for 
writing  something  else,  and  0  points  for  writing  nothing  (there  were  12  points  max- 
imum).  Whole-analogies  were  scored  similarly  (3  points  maximum). 

The  responses  to  the  test  problem  were  evaluated  for  quantity  (each  solution  re- 
ceived 1  point)  and  quality.  The  quality  of  the  target  solution  was  evaluated  as  fol- 
lows: 2  points  for  radiation  from  different  directions;  1  point  for  low-intensity;  1 
point  for  simultaneity  (or  converging). 

We  each  independently  evaluated  responses  based  on  pre  determined  criteria. 
Overall,  there  were  11  differences  in  scoring  of  the  test;  5  were  readily  reconciled 
as  errors  or  oversights.  Pearson  product-moment  correlations  between  ratings 
were  then  calculated  to  estimate  interrater  reliability  for  the  number  of  solutions  (r 
=  .983)  and  for  the  score  on  the  test  (r  =  .987). 


Results 

We  focused  our  analysis  on  the  average  quality  of  the  solutions  to  the  radiation 
problem  (maximum  =  4).  Because  our  design  had  only  one  level  in  the  control 
group,  we  performed  two  analyses  of  variance:  (a)  a  one-way  5-group  ANOVA,  and 
(b)  a  two-way  ANOVA.  Hypothesis  1 — that  generating  summaries  or  analogies  for 
an  analog  stor>  would  facilitate  subjects'  abilities  to  generate  the  analogical  dis- 
persion solution  to  the  radiation  problem — was  not  supported.  Hypothesis  2 — that 
generating  summaries  or  analogies  for  each  part  of  the  analog  story  would  further 
facilitate  subjects'  abilities  to  generate  the  dispersion  solution  to  the  radiation  prob- 
lem— was  partially  supported.  Table  1  shows  the  mean  scores  of  the  solutions. 


Table  1.  Mean  Scores  of  Solutions  * 


Analogy 

Summary 

Reading  (control) 

Paragraph 

2.11  (n=14) 

1.75   (n  =  12) 

1.94   (n  =  26) 

Whole 

.89   (n  =  14) 

.68   (n  =  14) 

.79   (n  =  28) 

1.50   (n  =  28) 

1.17   (n  =  26) 

1.343    <n  =  54) 

.79  (n=14) 

1.22  (n  =  68) 

*  maximum  score  =  4 


The  one-way  ANOVA  indicated  that  the  differences  between  the  means  of  the 
five  groups  (Table  1)  approached  significance  (p  =  .069).  Because  the  difference  ap- 
proached significance,  we  decided  to  compare  the  groups  as  we  had  initially 
planned.  A  Fisher  PSLD  post  hoc  test  revealed  statistically  significant  differences 
(p  <  .05)  between  the  paragraph-analogy  group  and  (a)  the  whole-analogy  group,  (b) 
the  whole-summary  group,  and  (c)  the  reading  group.  A  two-factor  ANOVA,  with 
the  control  group  data  omitted,  further  clarified  the  effects  of  the  treatments.  It  re- 
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vealed  a  main  effect  for  frequency  of  processing  (Table  1,  Paragraph  and  Whole); 
that  is,  subjects  who  responded  to  each  paragraph  with  an  analogy  or  a  summary 
had  a  statistically  higher  mean  score  (p  =  .013)  than  subjects  who  responded  at  the 
end  of  the  whole  story.  Although  the  analogizers  scored  higher  than  the  summariz- 
ers  (1.5  vs.  1.17),  there  was  no  main  effect  for  the  mode  of  processing.  Thus,  there 
was  a  clear  value  in  responding  to  the  story  paragraph  by  paragraph,  and,  from  the 
one-way  ANOVA,  possible  value  in  analogizing  rather  than  summarizing. 

Other  factors.  In  their  answers  to  the  test  problem,  7  subjects,  who  were  dis- 
tributed in  a  nonsignificant  pattern  across  treatments,  referred  to  the  analog  story. 
Six  of  those  subjects  (85.7%)  scored  at  least  3  on  the  test.  An  analysis  of  variance 
revealed  a  statistically  significant  difference  (p  =  .0001)  between  the  average  test 
scores  of  those  7  subjects  and  the  scores  of  the  61  subjects  who  did  not  refer  to  the  ana- 
log story  (3.5  vs.  .967). 

Quality  of  Processing 

Surprisingly,  the  quality  of  summaries  and  analogies  was  not  significantly 
related  to  solution  of  the  radiation  problem.  Quality  accounted  for  only  3  -  23%  of 
the  variance  in  all  treatments,  even  though  the  quality  of  subjects'  summaries  and 
analogies  varied  greatly. 

For  example,  the  subject  who  wrote  the  following  good  summary  scored  0  on  the 
problem: 

•  A  clever  military  leader  succeeds  in  overthrowing  a  ruthless  despot 
His  troops  evade  destruction  and  entrapment  by  attacking  the  central 
fortress  from  diverse  directions. 

The  range  in  the  quality  of  analogies  was  even  wider  than  for  summaries. 
Probably  the  poorest  analogy  was  the  following,  written  by  a  subject  who  scored  0  on 
the  problem: 

•  The  general  is  great  leader  to  make  dicision  quickly. 
Occasionally,  analogies  were  very  general  comparisons,  sometimes  so  broad  as  to 
seem  useless.  For  example: 


•  Dictator  is  like  many  kings  of  medieval  times.  Dictator  reminds  me  of 
Russian  czars.  The  great  general  is  like  Stalin. 

•  Capturing  the  fortress  is  like  solving  a  problem, 


•  This  reminds  me  of  two  things— the  Alamo  and  the  Ho  Chi  Min  trail. 
Yet  even  subjects  who  drew  direct  analogies,  sometimes  paragraph  by  paragraph,  to 
Desert  Storm  did  not  necessarily  get  the  correct  solution  to  the  problem  (the  Gulf 
War  ended  a  few  months  before  the  study  was  conducted).  For  example: 

(1)  Kuwait  fell  to  Saddam  Hussein,  he  ruled  from  Iraq-Kuwait. 

(2)  UN  forces  gathered  in  surrounding  countries — Scud  missiles  threat- 
ened &  mines  [were]  planted. 

(3)  Waiting  period  of  UN  troops  prior  to  attacking. 

(4)  UN  troops  attack  from  different  directions  with  few  casualties  & 
Kuwait  reclaimed. 


As  predicted,  cognitively  processing  the  story  in  parts  (paragraphs)  rather  than 
as  a  whole,  or  merely  rereading  it,  facilitated  solution  of  the  test  problem.  This 
finding  is  consonant  with  the  theory  of  deep  processing  (Craik  &  Lockhart,  1972), 
as  well  as  with  the  generative  model  of  learning  (Wittrock,  1974)  and  the  theory  of 
assimilative  encoding  (Mayer,  1980). 

Although  the  pattern  of  scores  was  in  the  expected  direction  (analogizing  and 
summarizing  greater  than  reading),  the  precise  reason  for  the  failure  to  find  a 
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main  effect  for  mode  of  processing  cannot  be  determined  from  the  data.  However, 
it  seems  likely  that  the  subjects  were  not  adequately  skilled  in  analogizing  and 
summarizing.  We  generally  assumed  that  graduate  students  would  possess  ade* 
quate  skills  for  both;  yet  their  responses  clearly  indicated  a  wide  range  of  skill  lev- 
els, particularly  in  analogizing.  This  variability  in  skill  levels  hurt  the  internal 
validity  and  control  of  the  study,  but  at  the  same  time  served  to  strengthen  the 
study's  external  validity.  Clearly  university-level  instruction  must  meet  the 
needs  of  students  with  widely  varying  skills. 

Another  reason  for  the  failure  to  find  a  main  effect  for  the  task  (i.e.,  mode  of 
processing)  might  be  lack  of  adequate  incentive.  Hicken,  Sullivan,  and  Klein 
(1992)  found  that  incentive  is  a  significant  factor  in  subjects'  performance  in  re- 
search studies.  Future  studies  should  provide  improved  incentives  for  solving  the 
problem. 

Although  the  paragraph-analogy  treatment  produced  the  best  results,  the  poten- 
tial effect  might  have  been  inhibited  not  only  by  an  inadequate  incentive  but  also  by 
other  factors.  For  example,  some  subjects  in  the  paragraph-analogy  treatment  ac- 
tually created  partially  or  fully  "connected"  analogies  (as  in  the  whole-analogy 
treatment)  rather  than  "unconnected"  analogies  (Simons,  1984)  and  may  thus  have 
inhibited  the  hypothesized  creative  transfer  of  knowledge.  Seven  subjects  (50%)  in 
this  treatment  drew  at  least  one  analogy  to  Desert  Storm,  particularly  to  Saddam 
Hussein  and  General  Schwartzkopf;  only  1  of  the  4  who  drew  more  than  1  compari- 
son to  Desert  Storm  got  the  correct  solution.  It  is  possible  that  subjects  using  Desert 
Storm  as  a  "connected"  analogy  strongly  coded  the  story  simply  as  a  "war  story," 
thus  making  transfer  to  another  problem  less  likely.  As  noted  above, 
"unconnected"  analogies  facilitate  creative  transfer  of  knowledge,  while 
"connected"  analogies  facilitate  storing  information  (Stepich  &  Newby,  1988). 

The  ability  to  identify  The  Attack-Dispersion  Story  as  an  analog  to  the  problem 
clearly  facilitated  solving  the  problem.  But  the  data  in  the  study  provide  no  insight 
into  why  some  subjects  were  able  to  make  the  connection.  The  6  who  referred  to  the 
story  and  scored  at  least  3  were  dispersed  across  treatments;  and  they  varied  in  age 
(34-52),  sex  (3  female,  3  male),  undergraduate  major  (art,  English,  electrical  engi- 
neering, education,  economics).  It  is  possible  that  others  were  aware  of  the  connec- 
tion but  just  did  not  mention  in  their  solutions  Future  studies  should  include  de- 
briefing of  subjects  to  determine  if  they  were  aware  of  the  connection. 

In  spite  of  the  limitations  of  the  study,  the  groups  which  processed  parts  rather 
than  the  whole  scored  higher  on  average  than  the  other  groups,  with  the  paragraph- 
analogy  group  scoring  higher  than  the  paragraph-summary  group.  Theory  does 
suggest  an  explanation  for  these  differences.  The  first  explanation  derives  di- 
rectly from  schema  theory's  explanation  of  semantic  networks.  These  networks 

represent  [the]  knowledge  structures  which  enable  learners  to  combine 
ideas,  infer,  extrapolate  or  otherwise  reason  from  them.  Learning  consists 
of  building  new  structures  by  constructing  new  nodes  and  interrelating 
them  with  existing  nodes  and  with  each  other.  The  more  links  that  the 
learner  can  form  between  existing  knowledge  and  new  knowledge,  the  bet- 
ter the  learner  will  comprehend  the  information  and  the  easier  learning 
will  be.  (Jonassen,  Cole,  &  Bamford,  1992,  p.  395) 
It  is  assumed  that  analogizing  and  summarizing  engage  our  semantic  networks 
more  than  passive  reading  does.  But  this  assumption  does  not  explain  why  the 
whole-analogy  and  whole-summary  treatments  performed  essentially  the  same  as 
the  re-read  group. 

We  believe  one  explanation  lies  in  the  fact  that  paragraph-level  processing 
tends  to  result  not  only  in  more  discrete  processing,  but  also  in  less  "packaged"  pro- 
cessing. That  is,  the  "unconnected"  analogies  and  summaries  make  the  knowl- 
edge more  accessible  for  far  transfer  than  do  "connected"  analogies  and  sum- 
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maries.  This  explanation  is  consonant  with  cognitive  flexibility  theory  (e.g., 
Spiro  et  al.  1988)  and  it  extends  explanations  of  "unconnected"  (conceptual)  versus 
"connected*  (comprehensive)  analogies  (e.g.,  Simons,  1984;  Stepich  &  Newby, 
1988)  to  summaries . 

A  final  question  is  why  the  "unconnected"  analogies  tended  to  produce  better 
performance  than  the  "unconnected"  summaries.  Since  both  treatments  were  in- 
tended to  produce  meaningful  encoding,  this  difference  is  not  readily  explainable. 
One  explanation  seems  particularly  plausible.  Because  analogizing  is  a  less  fa- 
miliar study  strategy  it  may  have  caused  the  learners  to  attend  to  details  more  care- 
fully and  thus  made  them  more  accessible  during  the  test.  However,  this  explana- 
tion as  well  as  others  must  be  explored  in  future  research. 


Implications  for  Instruction 

The  most  important  implications  relate  to  focus  and  types  of  processing: 
Instructional  strategies  that  engage  the  learner  in  generating  summaries  and 
analogies  in  the  learner's  own  words  of  subcomponents  rather  than  just  main 
ideas  tend  to  facilitate  far  transfer  problem  solving,  at  least  in  immediate  situa- 
tions. This  finding  is  consonant  with  Wittrock  and  Alesandrini's  (1990)  findings 
and  with  the  generative  model  of  learning  in  general  (Wittrock,  1974).  It  is  also 
consonant  with  theories  of  conceptual  versus  comprehensive  analogies  (e.g., 
Stepich  &  Newby,  1988),  and  with  cognitive  flexibility  theory  (e.g.,  Spiro  et  al., 
1988),  which  argues  for  the  use  of  multiple  analogies  in  instruction.  Whether  these 
findings  apply  to  other  domains,  to  delayed  transfer,  and  to  the  study  of  longer  text, 
and  whether  there  are  any  aptitude-strategy  interactions  remain  to  be  investigated. 
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Appendix 

The  Story 

1  A  small  country  fell  under  the  iron  rule  of  a  dictator.  The  dictator  ruled  the 
country  from  a  strong  fortress.  The  fortress  was  situated  in  the  middle  of  the 
country,  surrounded  by  farms  and  villages.  Many  roads  radiated  outward 
from  the  fortress  like  spokes  on  a  wheel. 

2  A  great  general  arose  who  raised  a  large  army  at  the  border  and  vowed  to 
capture  the  fortress  and  free  the  country  and  the  dictator.  The  general  knew 
that  if  his  entire  army  could  attack  the  fortress  at  once  it  could  be  captured.  His 
troops  were  poised  at  the  head  of  one  of  the  roads  leading  to  the  fortress,  ready  to 
attack.  However,  a  spy  brought  the  general  a  disturbing  report.  The  ruthless 
dictator  had  planted  mines  on  each  of  the  roads. 

3  The  mines  were  set  so  that  small  bodies  of  men  could  pass  over  them  safely, 
since  the  dictator  needed  to  be  able  to  move  troops  and  workers  to  and  from  the 
fortress.  However,  any  large  force  would  detonate  the  mines.  Not  only  would 
this  blow  up  the  road  and  render  it  impassable,  but  the  dictator  would  then  de- 
stroy many  villages  in  retaliation.  A  full-scale  direct  attack  on  the  fortress 
therefore  appeared  impossible. 

4  The  general,  however,  was  undaunted.  He  divided  his  army  into  small 
groups  and  dispatched  each  group  to  the  head  of  a  different  road.  When  all  was 
ready  he  gave  the  signal,  and  each  group  charged  down  a  different  road.  All  of 
the  small  groups  passed  safely  over  the  mines,  and  the  army  then  attacked  the 
fortress  in  full  strength.  In  this  way  the  general  was  able  to  capture  the  fortress 
and  overthrow  the  dictator. 


Test  Directions 


Before  you  take  a  test  to  see  what  you  learned  from  the  story,  we  would  like  you  to 
try  to  solve  the  following  problem: 

Suppose  you  are  a  doctor  faced  with  a  patient  who  has  a  malignant  tumor  in 
his  stomach.  It  is  possible  to  operate  on  the  patient,  but  unless  the  tumor  is  de- 
stroyed the  patient  will  die. 

There  is  a  kind  of  ray  that  can  be  used  to  destroy  the  tumor.  If  the  rays  reach 
the  tumor  all  at  once  at  a  sufficiently  high  intensity,  the  tumor  will  be  de- 
stroyed. Unfortunately,  at  this  intensity  the  healthy  tissue  that  the  rays  pass 
through  on  the  way  to  the  tumor  will  also  be  destroyed.  At  lower  intensities  the 
rays  are  harmless  to  healthy  tissue,  but  they  will  not  affect  the  tumor  either. 

Without  cutting  the  patient,  what  type  of  procedure  might  be  used  to  destroy 
the  tumor  with  the  rays,  and  at  the  same  time  avoid  destroying  the  healthy  tis- 
sue? 

In  the  space  below,  /rite  down  as  many  procedures  as  you  can  think  of.  If 
you  need  extra  space,  you  may  write  on  the  back. 
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Introduction 

Traditional  approaches  to  instructional  design  involve  needs  analysis,  selection 
and  sequencing  of  instructional  strategies,  and — if  time  and  budget  allow — an  itera- 
tive process  of  formative  evaluation  and  revision  until  the  instructional  package 
meets  some  pre-determined  criterion  for  success.  There  are  many  problems  to  such 
approaches,  not  the  least  of  which  is  the  fact  that  formative  evaluation  often  gets 
short  shrift  because  of  time  and  budget  constraints.  But  a  larger  problem  derives 
from  the  behaviorist  roots  of  instructional  design.  Winn  (1990)  noted  that  ap- 
proaches which  are  grounded  in  behavioral  theory  are  "inadequate  to  prescribe  in- 
structional strategies  that  teach  for  understanding*  (p.  53;  see  also,  fur  example, 
Bednar,  Cunningham,  Duffy,  &  Perry,  1991).  Winn  (1990)  argued  that  "design  needs 
to  be  integrated  into  the  implementation  of  instruction"  (p.  53). 

In  this  paper,  I  propose  one  strategy  that  integrates  design  into  the  implementa- 
tion of  instruction.  Specifically,  I  argue  that  increased  use  of  learner-generated  ques- 
tions and  comments  can  benefit  every  stage  of  the  instructional  process — from  design 
through  implementation.  My  argument  assumes  that  instructional  strategies  can  do 
double  duty.  First,  they  can  adapt  instruction  to  students'  needs  "on  the  fly,"  and 
second,  they  can  provide  ongoing  formative  evaluation. 

In  the  first  part  of  the  paper,  I  show  how  learner-generated  questions  and  com- 
ments are  an  instance  of  a  larger  trend  toward  constructivism.  Second,  I  describe 
one  use  of  journals  to  elicit  and  respond  to  questions  and  comments.  Third,  I  discuss 
the  potential  of  learner-generated  questions  and  comments  in  formative  evaluation. 
Fourth,  I  discuss  implications  for  instruction  and  learning-  Finally,  I  offer  recom- 
mendations for  incorporating  learner-generated  questions  and  comments  in  the  in- 
structional design,  development  and  implementation  process. 

Limitations  of  Adjunct  Questions 

It  is  a  fairly  common  practice  across  disciplines  to  include  adjunct  questions  in 
the  students'  text,  courseware,  or  teachers'  guides.  Yet  the  research  findings  on  the 
use  of  adjunct  questions  have  often  been  contradictory,  with  outcomes  vaiying  on  a 
wide  variety  of  factors,  including  the  placement  of  questions,  level  of  questions,  and 
type  of  learning  outcomes  (e.g.,  Lindner  &  Rickards,  1985).  The  effectiveness  of  ad- 
junct questions  is  also  challenged  by  the  literature  on  individual  differences  such  as 
motivation,  prior  knowledge,  field  dependence/independence.  For  example,  according 
to  Jonassen  and  Grabowski  (in  press),  the  structure  which  adjunct  questions  imposes 
should  benefit  field  dependent  students  while  interfering  with  the  learning  of  field 
independent  students,  who  are  likely  to  benefit  more  from  imposing  their  own  struc- 
ture on  material. 

Another  limitation  of  adjunct  questions  is  that  they  identify  problems  for  learners 
rather  than  developing  the  learner's  ability  to  identify  problems.  Many  (e.g.,  Brown, 
Campione,  &  Day,  1981;  Duffy  &  Jonassen,  1992;  Frase  &  Schwartz,  1975;  Gagn6, 
1980;  Jonassen,  1985)  have  argued  that  the  goal  of  education  is  to  help  students  be- 
come independent  learners.  And  being  an  independent  learner  involves  not  only 
solving  but  identifying  the  problems  to  be  solved  (Bransford  &  Stein,  1984). 

In  this  regard,  research  has  generally  overlooked  an  instructional/learning  strat- 
egy potentially  more  important  than  adjunct  questions:  learner-generated  questions 
and  comments.  Depending  on  their  metacognitive  sophistication  and  cognitive  ori- 
enting tasks  (Rigney,  1978),  students  may  generate  such  questions  and  comments  as 
they  read  an  assignment,  complete  an  exercise  (including  answering  adjunct  ques- 
tions), participate  in  class  discussion,  study  for  an  examination,  etc.  Research  has 
shown  that  students'  comments,  answers  and  even  their  questions  can  influence  how 
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they  construct  knowledge.  For  example,  Chi  and  her  colleagues1  (Chi  &  Bassock, 
1989;  Chi,  Bassock,  Lewis,  Reimann,  &  Glaser,  1989;  Chi,  de  Leeuw,  Chiu,  & 
LaVancher,  1191;  Chi  &  VanLehn,  1991)  investigated  students'  self-explanations 
while  studying  worked-out  physics  examples.  Chan,  Burtis,  Scardamalia,  and 
Bereiter  (1992)  studied  how  students  learn  from  text.  And  Feathsrs  and  White 
(1987)  focused  on  students'  development  of  metacognitive  awareness  in  a  college  de- 
velopmental reading  class.  I  have  been  studying  students'  questions  and  comments 
in  journals  in  a  college  literature  course  (Cole,  1991,1992) . 

As  Stein  and  Bransford  (1979)  observed,  "An  emphasis  on  the  types  of  questions 
students  ask  themselves  may.. .have  important  implications  for  understanding  indi- 
vidual differences  in  learning  and  retention"  (p.*'  776).  Although  these  differences 
would  seem  to  have  important  implications  for  selection  and  sequencing  of  instruc- 
tion, the  traditional  instructional  design  and  development  process  pays  little  heed  to 
students'  questions  and  comments  until  the  formative  evaluation  process. 
(Unfortunately,  with  the  budgetary  and  time  constraints  typically  imposed  on  in- 
structional design  and  development,  formative  evaluation  often  gets  short  shrift,  if  it 
is  included  at  all  in  the  process  [Martin  Tessmer,  personal  communication,  1991].)  In 
fact,  the  traditional  instructional  design  and  development  process  often  seems  to  dis- 
count the  potential  of  students'  comments  and  questions.* 

A  Limitation  of  Experts 

One  of  the  problems  of  relying  mainly  on  experts  in  selecting  and  sequencing  in- 
struction is  that  they  find  it  difficult  to  decompose  their  knowledge,  even  with  the 
help  of  designers  and  developers.  Moreover,  they  are  often  out  of  touch  with  the 
needs  of  novices  and  may  hold  widely  disparate  views  of  the  learners'  needs. 

For  example,  in  a  study  of  students' journals  in  a  college  introduction  to  litera- 
ture class  (Cole,  1992),  I  had  planned  to  use  faculty  ratings  of  story  difficulty  to  guide 
my  selection  and  sequencing  of  stories.  During  the  first  week  of  the  term,  I  adminis- 
tered the  survey  to  the  other  six  full-time  members  of  the  college-level  English  pro- 
gram. Four  of  the  teachers  had  taught  full-time  at  the  college  more  than  20  years; 
the  fifth  had  taught  full  time  more  than  15  years;  the  sixth  had  taught  full  time  less 
than  2  years. 

In  spite  of  their  expertise,  the  survey  revealed  no  consensus.  The  faculty,  none  of 
whom  used  journals  in  their  classes,  did  not  agree  unanimously  on  the  rating  of  any 
story,  although  83%  (5  out  of  6)  did  agree  on  the  rating  of  one  story  (Faulkner's  The 
Bear"  is  Difficult).  The  faculty  ratings  for  29%  of  the  14  stories  with  which  they  were 
all  familiar  spanned  the  entire  range  (Easy,  Moderate,  Difficult).  Including  my  rat- 
ings for  those  stories  increased  the  disparity  to  36%.  The  range  of  responses  provides 
further  justification  of  the  usefulness  of  journals  in  communicating  students'  difficul- 
ties. If  teachers  with  so  much  experience  do  not  agree  on  the  difficulty  level,  then  at 
least  some  of  them  must  have  inaccurate  schemata  of  their  students'  needs. 

There  is  some  evidence  that  journals  help  "experts"  identify  the  needs  of  novices. 
My  ratings  agreed  unanimously  with  those  of  a  part-time  composition/literature 
teacher  who,  for  several  years,  has  required  her  students  to  write  journals.  A  follow- 
up  discussion  revealed  that  we  had  based  our  ratings  on  similar  information  we  had 
obtained  from  students'  journals  more  than  from  class  discussions. 


1For  example,  Merrill  (Gagne  &  Merrill,  1991)  recently  said,  "Once  in  a  while  I  get  a  good 
question,  but  most  of  the  time  I  don't.  Consequently,  I  don*t  think  the  students  know  what  to 
ask  anyway*  (p.  31).  Merrill  is  evaluating  questions  from  the  perspective  of  an  expert,  not 
from  the  perspective  of  a  novice  or  even  a  journeyman.  On  the  other  hand,  Scardamalia  and 
Bereiter  (1991,  p.  37)  found  that  even  "children  can  produce  and  recognize  educationally  pro 
ductive  questions  and  can  adapt  them  to  their  knowledge  needs." 
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The  Paradigm  Shift 

The  emerging  interest  in  learner-generated  questions  has  followed  closely  on  the 
heels  of  the  paradigmatic  shift  in  psychology  from  behaviorism  to  constructivism 
(Bruning,  1983;  Resnick,  1983;  also  see  Duffy  &  Jonassen,  1992)  and  the  accompany- 
ing shift  from  mathemagenic  instruction,  which  aseek[s]  to  control  the  information 
processing  activities  of  the  learner"  to  generative  learning,  which  places  the  locus  of 
control  in  the  learner  (Jonassen,  1985,  p,  127).  The  generative/constructivist  learning 
model  is  consonant  with  encouraging  learners  to  become  independent  by  learning 
how  to  learn  (e.g.,  Jonassen,  1985). 

The  generative  model  asserts  that  learners,  when  faced  with  stimuli. .  .construct 
and  assign  meaning  to  that  information  based  upon  prior  learning"  (Jonassen,  1985, 
p.  11).  In  emphasizing  the  active  role  of  the  learner,  the  generative  model  has  bear- 
ing on  many  recent  focuses  in  instructional  theory  and  research,  from  problem-solv- 
ing, and  schema  theory,  which  explains  comprehension  in  terms  of  the  interaction  be- 
tween the  reader  and  the  text  (e.g.,  Anderson,  1977;  Rumelhart  &  Ortony,  1977),  to 
the  views  of  transactional  analysis,  which  is  applied  so  well  to  the  study  of  literature 
by  Rosenblatt  (1978).  More  recent  discussions  of  schemata  (e.g.,  Clancey,  1992)  em- 
phasize that  they  are  not  stored  mental  structures  but  dynamic  networks  which  are 
"constantly  changing  and  always  freshly  created  (albeit  it  out  of  previous  activa- 
tions)* (p.  153).  Similarly,  Spiro  and  his  colleagues  (e.g.,  Spiro,  Coulson,  Feltovich,  & 
Anderson,  1988;  Spiro  &  Jehng,  1990)  describe  a  process  of  schema  assembly 
(forming  new  schema  from  parts  of  many  schemata)  in  ill-defined  domains. 

As  active  learners,  "people  tend  to  generate  perceptions  and  meanings  that  are 
consistent  with  their  prior  learning"  (Wittrock,  1974,  p.  88)  and  with  their  "age,  sub- 
culture, experience,  education,  interests,  and  belief  systems"  (Anderson,  Reynolds, 
Schallert,  &  Goetz,  1977,  p.  378;  Bransford  &  Johnson,  1972),  The  process  seems  to 
require  not  only  constant  creation,  as  noted  above,  but  also  reception  of  information 
and  availability  and  activation  of  relevant  schemata  (as  described  by  Mayer,  1975, 
1979,  1980).  But,  as  Anderson  et  al.  (1977)  noted,  students  sometimes  do 
"violence.. .to  the  'data'  contained  in  the  text"  (p.  371).  Thus,  it  is  important  to  design 
and  develop  instruction  which  can  identify  problems  and  support  the  learning  process 
in  a  timely  manner— that  is,  to  help  current  students,  not  merely  to  improve  instruc- 
tion for  future  students. 

Learner-generated  Question*  and  Comments 

One  instructional  intervention  which  seems  particularly  promising  focuses  on 
learner-generated  questions  and  comments.  The  limited  research  on  learner-gener- 
ated questions  has  focused  primarily  on  training  learners  to  generate  the  type  of 
questions  that  teachers  will  ask  on  tests  and  examinations  (e.g.,  Andre  &  Anderson, 
1978-79;  Frase  &  Schwartz,  1975)  or  on  studying.  This  approach  generally  assumes 
a  common  level  of  learner  understanding  and  thus  fails  to  address  the  individual 
learner's  prior  knowledge  and  misconceptions.  Another  approach  is  to  study  the  re- 
lationship between  question  frequency  and  individual  differences,  such  as  prior 
knowledge  (e.g.,  Miyake  &  Norman,  1979). 

However,  focusing  narrowly  on  "questions"  overlooks  an  intuitively  more  natural 
strategy  of  teaching  students  to  articulate  and  reflect  on  the  questions  and  comments 
they  have  as  they  try  to  master  a  domain.2  The  student's  own  questions  and  com- 
ments might  paradoxically  be  viewed  as  enlarging  what  Vygotsky  (1978)  called  the 


The  cognitive  apprenticeship  model  (Collins,  Brown,  &  Newman,  1989)  is  exploring  new 
territory,  but  does  not  directly  address  thin  issue. 
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zone  of  proximal  development.3  When  coupled  with  other  instructional/learning 
strategies,  such  as  journal  writing  or  question  sessions,"*  this  strategy  allows  the 
learner  to  build  more  naturally  on  whatever  prior  knowledge  he/she  has,  while  pro- 
viding an  opportunity  for  the  teacher  to  monitor  the  student's  comprehension,  iden- 
tify misconceptions,  and  adjust  instruction  accordingly.^  In  anonymous  evaluations 
of  journal  writing,  several  of  my  students  have  commented  that  the  act  of  writing 
journal  questions  helped  them  focus  their  attention  and  sometimes  helped  them  find 
the  answer  themselves  (Cole,  1991,  1992).  Such  an  approach  is  a  departure  from 
traditional  instructional  design  and  development.  Fostering  students'  questions  and 
comments  assumes  that  the  selection  and  sequence  of  instruction  cannot  be  totally 
predetermined  and  that  it  requires  adjustment  "on  the  fly." 

A  domain-specific  strategy?  Research  must  determine  whether  learner-gen- 
erated questions  and  comments  are  more  critical  in  some  domains  than  in  others. 
However,  identifying  bugs  in  procedural  knowledge  such  as  arithmetic  seems  easier 
(not  necessarily  easy)  than  in  ill-defined  domains  such  as  understanding  complex  col- 
lege-level short  stories.  In  arithmetic,  the  teacher  or  the  student  can  readily  deter- 
mine if  the  student  has  arrived  at  a  wrong  answer  to  a  problem;  and  the  teacher  can 
identify  the  errors)  by  having  the  student  think  aloud  while  working  through  a  simi- 
lar problem.  Because  the  procedure  for  a  given  problem  is  clear  cut,  a  computer  pro- 
gram can  even  monitor  students'  actions,  intervene,  provide  guidance,  etc. 

Teaching  college-level  short  stories  is  much  more  complicated.  First,  a  story  may 
not  have  a  single  "correct"  meaning;  thus  the  teacher  ideally  should  know  not  only 
the  student's  conclusion,  but  also  his/her  reasoning.  Similarly,  the  student  may  have 
inferred  a  "correct"  meaning  from  wrong  or  incomplete  evidence.  While  solving 
arithmetic  requires  knowledge  of  a  well-defined  procedure,  understanding  a  complex 
story  is  a  dynamic  process  in  which  the  meaning  unfolds  with  each  word,  each 
phrase,  each  sentence,  etc.  The  student  may  have  a  weak  understanding  of  grammar 
(from  pronoun  use  to  participial  phrases),  or  may  miss  a  symbol,  an  allusion  or  irony. 
Moreover,  the  author  often  delay*  or  even  withholds  information  for  artistic  or  affec- 
tive purposes.  The  potential  number  of  errors  is  infinite. 

While  students'  journals  cannot  provide  insight  into  every  error,  they  often  pro- 
vide insight  into  critical  problems,  which  the  teacher  can  explore  in  class  or  in  tuto- 
rial sessions. 

Learning  to  Learn 

Developing  the  ability  to  monitor  one's  comprehension,  articulate  questions,  and 
explore  answers  requires  time  and  practice.  Writing  journals  as  homework  gives 
students  adequate  time  to  reflect  and  to  try  to  answer  their  own  questions.  I  found 
(Cole,  1992)  that  my  students'  journals  contain  significantly  more  comments  than 


^Vygotsky  (1978)  defined  the  zone  of  proximal  development  as  "the  distance  between  the 
actual  development  level  as  determined  by  independent  problem  solving  and  the  potential  level 
of  development  as  determined  through  problem  solving  under  adult  guidance,  or  in  collabora- 
tion with  more  capable  peers"  (p.  86).  Compare  Bereiter's  (1985)  discussion  of  "the  learning 
paradox." 

4In  their  journals,  I  encourage  students  to  ask  any  question  or  write  any  comment  that 
they  believe  will  help  them  understand  the  reading  assignment  better  (with  the  exception  of 
definitions  of  words,  unless  they  are  unsure  of  the  appropriate  definition). 

In  lieu  of  at-home  journals,  at  the  beginning  of  a  discussion,  I  occasionally  ask  students  to 
writ*  the  three  questions  they  most  want  answered  about  the  reading  assignment;  then  I  col- 
lect them  and  read  them  anonymously  to  the  class.  Occasionally  I  juet  have  students  ask  their 
questions  out  loud  and  I  write  them  on  the  board. 

^If  many  students  raise  the  same  issues  in  their  journals,  then  I  respond  to  the  class  as  a 
whole.  If  only  a  few  students  raise  an  issue,  then  I  respond  in  each  journal. 
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questions,  apparently  because  once  students  identify  a  point  of  confusion  most  of 
them  try  to  resolve  it.  As  noted  above,  students  often  comment  that  the  act  of  writing 
their  questions  often  helps  them  discover  the  answers.  Journal  writing  seems  to  help 
students  expand  their  own  zones  of  proximal  development,  making  their  tacit  inter- 
pretations explicit  and  thus  available  for  reflection  and  analysis.  Thus  journal  writ- 
ing seems  to  be  an  instructional  approach  that  can  help  learners  make  the  transition 
from  being  dependent  on  others  to  being  able  to  learn  on  their  own.  It  helps  them 
move  from  what  Vygotsky  (1978)  called  the  other-directed  to  self-directed  stages  of 
understanding. 

Moreover,  journal  writing  gives  each  student  the  opportunity  to  ask  questions, 
try  to  answer  them,  and  express  his/her  understanding  of  a  reading  assignment.  A 
few  students  do  not  have  the  opportunity  to  come  up  with  all  the  questions  and  an- 
swers and  make  it  impossible  for  other  students  to  contribute  to  their  own  under- 
standing. (Even  if  they  have  the  opportunity  in  class  discussions  or  in  collaborative 
groups,  some  students  don't  take  advantage  of  it,  for  various  reasons,  many  of  which 
relate  to  lack  of  confidence  or  fear  of  peer  ridicule.)  In  describing  their  Jasper  series, 
the  Cognition  and  Technology  Group  at  Vanderbilt  (1992)  emphasized  the  importance 
of  each  student  having  the  opportunity  to  contribute  to  his^er  learning.  Journal 
writing,  even  more  than  class  discussions  and  collaborative  learning,  guarantees 
such  opportunities. 

Although  other  instructional  strategies  theoretically  can  elicit  students'  questions 
and  comments  (e.g.,  encouraging  students  to  ask  questions  in  class,  and  asking  stu- 
dents questions),  most  students,  at  least  those  in  a  community  college,  seem  much 
more  willing  to  share  these  privately  than  in  public.  (I  base  this  conclusion  on  25 
years  experience  as  a  college  composition  and  literature  teacher  as  well  as  on  student 
evaluations  of  journal  writing.)  Electronic  networking  provides  an  alternate  vehicle 
for  anonymous  sharing  of  questions  and  comments  (see  Hubbard  &  Duffy,  1991). 
Unlike  adjunct  questions,  learner-generated  questions  and  comments  run  little  risk 
of  overwhelming  low -ability  learners  or  boring  high-ability  ones. 


In  the  past  eight  years,  I  have  incorporated  journal  writing  as  an  instruc- 
tional/learning strategy  in  approximately  30  classes  of  literature  students  at 
Arapahoe  Community  College,  a  large,  public  community  college  in  a  white-collar 
suburb  of  Denver,  Colorado.  Composition/literature  instructors  teach  five  classes 
with  a  maximum  of  23  students.  Students  at  the  college  vary  greatly  not  only  in 
reading  ability6  but  also  in -age,  academic  majors,  world  knowledge,  and  motivation. 
More  than  half  of  the  students  work.  The  average  age  for  many  years  has  been  at 
least  28  (spring  1992  it  was  31.2),  but  the  distribution  is  bimodal,  with  the  majority 
of  the  day-time  students  being  traditional  college  age  and  the  majority  of  the  night 
students  being  non -traditional  age.  While  most  of  the  students  in  my  literature 
classes  read  at  or  above  the  Uth-grade  level,  occasionally  a  few  students  read  at  or 
below  the  9th-grade  level.  At  the  other  extreme,  sometimes  half  the  students  read  at 
or  above  the  16.9th-grade  level  (the  highest  level  on  the  exam). 

The  more  journals  I  read,  the  more  I  have  become  aware  of  the  disparity  between 
(a)  the  adjunct  questions  in  texts  and  teachers'  guides — resources  I  had  relied  on 


e  college  requires  all  students  who  e.  oil  in  composition  to  take  reading  and  English 
placement  tests.  In  addition,  I  usually  administer  the  Nelson-Denny  Reading  Test  (Form  E, 
1981  edition)  during  the  first  week  of  my  literature  classes. 
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whenever  I  taught  an  unfamiliar  work— and  (b)  the  types  of  questions  and  comments 
in  students' journals.  The  adjunct  questions  in  most  texts  and  instructor  guides  tend 
to  address  relatively  high-level  learning  outcomes  related  to  aesthetic  principles — for 
example,  synthesis  and  evaluation  on  Bloom's  taxonomy  (Bloom  et  al.,  1956)  and 
problem-solving  on  the  Gagn6-Briggs  taxonomy  (Gagn6,  Briggs,  &  Wager,  1988).7 
The  questions  and  comments  from  a  whole  class  generally  span  Bloom's  and  Gagn6- 
Briggs'  taxonomies,  but  students'  problem-solving  efforts  tend  to  focus  on  what 
Kintsch  (1989)  calls  the  text-base  and  the  situation  model  rather  than  on  aesthetic 
principles. 

With  such  a  range  of  questions  and  comments,  it  is  imperative  to  be  able  to  iden- 
tify and  respond  to  individual  student  needs;  yet  the  nature  of  community  college 
students  presents  particular  challenges,  in  addition  to  the  range  in  their  reading  abil- 
ities. Students  often  lack  confidence  to  participate  in  class  discussion  (even  if  they 
read  at  the  highest  grade  level).  Thus  if  a  teacher  relies  on  adjunct  questions  or 
teachers'  guides  to  select  and  sequence  instruction,  he/she  is  likely  to  talk  over  the 
heads  of  many  students  and  bore  others. 

.Journal  Methodology 

Instructions.  Following  are  the  instructions  printed  on  the  syllabus  of  my 
Introduction  to  Literature  course  (I  refine  the  directions  from  time  to  time;  I  use  es- 
sentially the  same  instructions  in  other  literature  classes): 

Write  a  journal  for  each  story,  at  least  a  half-page  long— but  there  is  no 
maximum  length.  Write  your  journal  in  ink,  using  complete  sentences— oth- 
erwise I  will  not  be  able  to  understanu  what  you  are  asking  or  saying.  Write 
any  questions  you  have — except  the  definition  of  a  word,  unless  you  ure  un- 
sure of  the  relevant  definition — or  any  comments  you  would  like  to  make 
about  the  story.  The  journal  is  not  busy  work,  but  is  intended  to  help  me 
know  what  you  need  help  with  in  understanding  the  story.  Write  any  ques- 
tions you  believe  will  help  you  understand  the  story  now  as  well  as  late*,  for 
the  examination — or  any  other  questions  you  are  just  curious  about. 
Journals  are  worth  a  maximum  of  10  points  each.  If  a  journal  meets  the  min- 
imum requirements,  you  will  receive  7.5  points — a  grade  of  C;  you  will  receive 
additional  credit,  depending  on  how  much  your  questions  and  comments  indi- 
cate that  you  are  paying  attention  to  all  the  elements  as  you  try  to  under- 
stand the  story.  There  are  no  wrong  questions  or  comments  (but  there  are 
questions  and  comments  which  ignore  the  details  of  a  story).  Journals  are  a 
tool  for  you  to  learn  about  what  you  don't  know,  not  to  show  me  that  you  al- 
ready know  everything.  Even  if  you  don't  understand  a  story  at  all  but  ask 
questions  about  what  you  don't  understand  and  indicate  that  you  are  paying 
attention  to  details,  you  can  receive  a  10. 

I  will  collect  your  journals  at  the  beginning  of  the  class  on  which  they  are 
due.  If  you  miss  that  class,  you  may  submit  your  journal  at  the  beginning  of 
the  next  class  you  attend. 
Practice  and  resources*  In  addition  to  these  instructions,  I  usually  have  stu- 
dents practice  writing  journals  on  a  haiku  poem  in  class  and  I  read  those  anony- 
mously to  the  class.  (I  begin  with  a  haiku  because  it  is  easy  to  incorporate  into  one 
class  session.)  Then  I  use  students'  comments  and  questions  as  a  basis  for  helping 
them  understand  the  poem.  Before  students  write  their  first  out-of-class  journal,  I 
provide  written  descriptions  of  the  processes  three  former  students  used  in  writing 


7Since  most  introduction  to  literature  texts  are  organized  by  topic  (e.g.,  characterization, 
theme,  and  symbolism),  adjunct  questions  in  the  student*1  text  generally  focus  on  that  topic  in 
a  single  work.  There  are  few  if  any  questions  to  direct  students*  understanding  of  the  other 
topics  for  that  work.  Moreover,  most  of  the  works  have  no  questions  at  all. 
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their  journals,  and  a  sample  journal  The  last  two  semesters  students  have  also  had 
access  to  HyperCard  tutorials  I  developed  on  writing  journal  questions  and  writing 
journal  comments  (I  am  experimenting  with  requiring  students  to  view  the  pro- 
grams). After  I  return  the  first  journals,  we  discuss  any  concerns  the  students  have 
about  writing  journals.  I  encourage  students  to  ask  questions  about  the  journal-writ- 
ing process  at  any  time. 

Grading.  I  grade  students' journals  and  return  them  at  the  next  class,  respond- 
ing in  writing  and/or  in  class  discussion.  Some  writers  have  advocated  that  journals 
not  be  graded;  others  (e.g.,  Holland,  1989;  Roth,  1985)  have  said  they  found  grading  a 
necessary  incentive.  Still  others  have  identified  graded  journal  writing  as  an  incen- 
tive for  students  to  read  assignments  and  reflect  on  them  before  coming  to  class  (e.g., 
Bauso,  1988;  Schwartz,  1989)  and  a  means  of  increasing  students'  confidence  in  their 
understanding  of  the  domain  (e.g.,  Zuevcher,  1989).  Advice  on  grading  and  feedback 
abound  (e.g.,  Bauso,  1988;  Myers,  1988;  Roth,  1985),  yet  no  one  has  provided  empiri- 
cal support  for  particular  recommendations. 

Maintaining  anonymity.  Although  some  teachers  (e.g.,  Fulwiler,  1989;  Heath, 
1988;  Wilson,  1989)  have  students  share  their  journals  with  the  class,  I  always  main- 
tain the  anonymity  of  students  because  I  found  (Cole,  1992)  that  on  evaluations  of 
journal  writing  some  students  objected  to  sharing  even  though  they  had  not  objected 
when  I  gave  them  the  opportunity  not  to  share  their  journals  in  groups.  Moreover, 
students  apparently  censure  what  they  write  when  they  know  other  students  will  be 
the  audience  (Cole,  1992). 

Adjunct  Questions  Versus  Student  Journals  on  One  Story 

A  comparison  of  adjunct  questions  and  student  journals  on  one  story  illustrates 
the  superiority  of  the  latter  in  addressing  students'  needs.  Below  I  compare  typical 
adjunct  questions  on  "The  Cask  of  Amontillado,"  by  Edgar  Allan  Poe,  with  excerpts 
from  students' journals;  the  text  my  students  use  does  not  have  adjunct  questions  on 
this  story.  I  usually  teach  this  story  after  students  have  written  several  journals. 
First  I  list  some  of  the  adjunct  questions  from  one  text  (Table  1)  and  explain  the 
weaknesses  of  the  questions  in  addressing  students'  needs.  Then  I  provide  excerpts 
from  students' journals  to  illustrate  students'  needs  (Table  2). 


Table  1.  Adjunct  Questions  on  The  Cask  of  Amontillado"  from  a  Student  Text 


1.  How  does  Montresor's  opening  paragraph  explain  the  conflict  in  the  story  and 
foreshadow  its  resolution? 

2.  How  does  Montresor's  apparent  concern  for  Fortunato's  health  enhance  the  sus- 
pense of  the  inevitable  climax. 

3.  How  does  the  "supreme  madness  of  the  carnival  season"  provide  an  appropriate 
environment  for  Montresor  to  initiate  his  plot? 

4.  Why  does  Montresor  wait  a  half-century  to  tell  his  story?  


Critique  of  Questions  in  Table  1.  The  questions  in  Table  1  reflect  the  decom- 
position of  the  story  by  an  expert.  While  these  are  ultimately  worthy  questions,  they 
do  not  address  the  immediate  needs  of  most  students  as  they  try  construct  a  basic 
understanding  of  the  story  and  of  the  classic  concepts  of  fiction  (e.g.,  conflict). 
Following  is  a  brief  analysis  of  the  problems  students  would  typically  encounter  in 
trying  to  answer  the  adjunct  questions  (numbers  refer  to  the  questions  above). 
1.  Most  students  do  not  understand  the  text-base  of  the  opening  paragraph,  particu- 
larly the  following  sentences: 
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I  must  not  only  punish,  but  punish  with  impunity.  A  wrong  is  unredressed 
when  retribution  overtakes  its  redresser.  It  is  equally  unredressed  when  the 
avenger  fails  to  make  himself  felt  as  such  to  him  who  has  done  the  wrong. 
Students  typically  need  help  in  paraphrasing  these  sentences;  in  fact,  meaningful 
discussion  of  the  story  must  begin  by  helping  the  student  understand  the  opening 
paragraphs — otherwise  students  cannot  meaningfully  anchor  class  discussion. 
Students  also  have  difficulty  identifying  the  conflict  in  a  story.  Before  we  ask 
students  to  "explain  the  conflict... and  foreshadow  its  resolution,"  we  should  make 
sure  they  can  identify  the  conflict  (as  well,  of  course,  as  understanding  the  con- 
cept of  conflict  in  a  story). 

2.  Many  students  miss  the  irony  of  Montresor' s  concern  and  most  are  unable  to 
identify  the  point  of  climax  when  I  raise  that  i*aue  in  class. 

3.  Most  students  have  a  single,  inappropriate  schema  for  "carnival  season,"  one  re- 
lated, for  example,  to  carnivals  in  shopping  malls  during  ihe  summer  rather  than 
to  Mardi  Gras.  Moreover,  many  of  my  students  do  not  even  know  what  Lent  is. 
None  of  the  adjunct  questions  in  the  texts  I  examined  addressed  two  critical  pas- 
sages. Many  students  ask  about  the  first  one  in  their  journals.  Because  the  signifi- 
cance of  the  reference  to  the  masons  depends  on  complex  world  knowledge,  its  com- 
plete significance  always  stumps  students  even  if  they  understand  the  double  enten- 
dre (it  also  eludes  most  literary  critics). 

...His  [Fortunato'sl  eyes  flashed  with  a  fierce  light  He  laughed  and  threw  the 
bottle  upward  with  a  gesticulation  I  did  not  understand. 

I  [Montresor]  looked  at  him  in  surprise.  He  repeated  the  movement — a 
grotesque  one. 

"You  do  not  comprehend?"  he  said. 

"Not,  I,"  I  replied. 

Then  you  are  not  of  the  brotherhood." 
"How?" 

"You  are  not  of  the  masons." 
"Yes,  yes,"  I  said;  "yes,  yes," 
"You?  Impossible!  A  mason?" 
"A  n.jison,"  I  replied, 
"A  sign  "  he  said. 

"It  is  this,"  I  answered,  producing  a  trowel  from  beneath  the  folds  of  my 
roquelaire. 

"You  jest,"  he  exclaimed,  recoiling  a  few  paces, ... 
Nor  does  any  of  the  texts  address  the  following  exchange  which  occurs  during  the 
climax,  as  Montresor  is  completing  his  live  entombment  of  Fortunato: 

"Yes,"  1 1  Montresor)  said,  "let  us  be  gone." 

"For  the  love  of  God,  Montresor!"  [emphasis  in  the  original] 

"Yes,"  I  said,  "for  the  love  of  God!" 
Only  the  most  astute  students  note  this  passage  in  their  journals,  although  class  dis- 
cussion indicates  that  they  rarely  understood  its  significance.  Note  in  Table  2  that 
one  student  not  only  n6ted  the  passage  but  appeared  to  understand  its  significance 
(see  Story  Element  "God,  crisis").  However,  since  she  had  already  studied  the  story 
in  another  course,  it  was  impossible  to  know  if  this  was  her  own  perception.  She  was 
one  of  the  best  readers  in  her  class  and  wrote  the  best  journals  on  each  assignment. 

Journal  Entries.  The  questions  in  Table  2  provide  a  strong  contrast  to  the  ad- 
junct questions.  I  have  provided  excerpts  to  illustrate  the  diversity  of  issues  and 
comprehension.  A  comprehensive  listing  is  outside  the  scope  of  this  paper.  Students' 
journals  on  this  story  spring  semester  1992  ranged  from  146  to  1,173  words,  with  a 
median  of  721  and  a  mean  of  644. 
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Table  2.  Excerpts  from  Students'  Journals  on  'The  Cask  of  Anion  till  ado" 


Story  Element 

Journal  Excerpt 

Carnival,  setting 

I  can't  recall  exactly  what  the  carnival  season  is  all  about 
Isn't  it  something  to  do  with  the  Easter  season? 

Character 

It  seems  that  all  of  the  narrator's  false  concern  could  be 
picked  up  by  Fortunato. 

Character 

The  story  never  mentioned  what  the  real  motive  for  the 
killing  of  Fortunato  [was].  Was  Montresor  justified?  Was  he 
just  plain  crazy? 

Character 

Why  did  Fortunato  trust  Montresor  enough  to  go  down  there 
if  he  was  such  a  bad  guy  and  did  so  many  rotten  things  to 
Montresor? 

Character 

Why  did  Montresor  have  doubts  about  the  Amontillado? 

Character 

Why  was  Fortunato  laughing  when  he  was  being  "walled 
up"? 

Character,  plot 

I  was  sure  Montresor  was  just  playing  a  joke  on  Fortunato. 

Character,  plot 

The  narrator  makes  a  big  deal  about  being  concerned  with 
Fortunato's  health,  and  asks  him  several  times  to  leave  the 
catacombs  to  get  away  from  the  damp  cellars,  not  to  aggra- 
vate Fortunato's  cough.  Why  would  he  constantly  offer  to  let 
the  man  out? 

Character,  plot 

What  or  why  does  this  man  [Montresor]  want  revenge? 

Climax 

What  is  the  climax? 

God,  crisis 

The  moment  of  greatest  tension  is  when  he  is  fitting  in  the 
last  &  eleventh  final  stone  in  the  hole.  What  is  at  stake? 
They  say,  "For  the  love  of  God."  Are  they  going  through  this 
for  the  love  of  God?  What  does  this  line  actually  refer  to? 
Could  it  be  that  God  poses  a  threat  to  one  of  these  two  men? 

Irony,  Setting 

I  also  think  there  is  some  ironic,  wry  humor  concerning 
Fortunato  asking  the  narrator  if  he  is  a  mason  and  the  nar- 
rator produces  a  trowel  as  confirmation. 
It  has  occurred  to  me  that  such  a  reference  as  "supreme 
madness"  of  carnival  and  the  ending  of  the  story  may  leave 
open  the  question  of  whether  this  tale  is  an  illustration  of  a 
mad  person. 

Luchesi 

I  don't  understand  what  the  character  Luchesi  has  to  offer 
the  story.... 

Luchesi 

The  narrator  is  very  clever  in  the  way  he  gets  Fortunato  to 
follow  him  into  the  vaults.  He  provides  many  excuses  as  to 
why  Fortunato  shouldn't  necessarily  go:  too  much  trouble,  he 
has  a  cold,  [Montresor]  has  an  engagement  with  Luchesi. 

Mason,  Character 

I  wonder  if  Fortunato  was  stupid  or  just  too  drunk  to  ques- 
tion "the  writing  on  the  wall."  Why  would  Montresor  have  a 
trowel  under  his  cloak?  Not  because  he  was  a  "Mason."  (This 
student  apparently  understood  the  double  entendre.] 
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Masons 

After  I  read  this  about  3  times  and  was  writing  my  first  jour- 
nal it  dawned  on  me  that  mason  in  this  conversation  means 
two  things.  Fortunato  is  talking  about  a  group  of  people 
called  the  Masons,  a  brotherhood  just  like  there  are  groups 
today  such  as  the  shriners,  or  the  clan  even  though  they  are 
completely  different  types  of  groups,  it  appears  Fortunato  is 
talking  about  a  fighting  group  the  masons.  Because  again  in 
the  beginning  of  the  story,  the  author  tells  us  he  was  a 
strong,  respected  man.  But  not  a  man  you  could  trust.  The 
other  kind  of  mason  is  the  stone  mason  and  the  stone  mason 
is  what  the  narrator  is. 

Masons 

I  am  not  sure  what  the  masons/brotherhood  had  to  do  with 
this,  but  the  narrator's  production  of  the  trowel  reminds  me 
of  yet  another  pun:  trowel  used  for  building  and  masonry. 

Masons,  irony 

One  of  my  favorite  elements  of  irony  in  the  story  is  when 
Fortunato  asks  if  Montresor  is  a  mason  he  is  speaking  of  the 
brotherhood.  [But)  When  Montresor  answers  he  is  speaking 
of  a  stone  mason  since  he  intends  to  wall  Fortunato  up. 

Nitre 

At  the  end  of  the  story,  "My  heart  grew  sick — on  account  of 
the  dampness  of  the  catacombs."  If  nitre  seeps  through  the 
walls,  along  with  the  chill  of  the  air  through  the  walls, 
[Montresor]  could  have  been  feeling  pretty  bad  physically. 

Nitre,  setting 

The  nitre  on  the  walls  was  interesting.  I  am  not  sure  if  I  see 
the  connections  unless  the  narrator  was  trying  to  tie  the  ex- 
plosivenesa  of  the  nitre  (potassium  nitrate  used  in  gunpow- 
der) to  the  explosiveness  of  the  situation. 

Nitre,  symbolism 

The  walls  are  covered  with  nitre.  There  is  mention  of  this 
several  times.  I  think  it  symbolizes  that  Montresor  believes 
that  Fortunato  "stinks"  as  a  human  and  is  also  a  "slime." 

Plot 

Why  did  they  wait  50  years  to  find  the  body?  [The  student 
mistakenly  believes  that  someone  found  the  body.] 

Plot  and  world 
knowledge 

What  was  the  horrible  gesture  that  Fortunato  made  in  the 
crypt  and  what  bearing  did  that  have  on  the  story? 

Semantic  meaning 

Impunity  was  used  a  lot  in  the  story,  but  I'm  not  sure  what 
the  meaning  is  in  relationship  to  the  story. 

Ex.  It  was  used  in  the  introduction  and  in  the  motto. 

Semantic  meaning 

What  does  Poe  mean  when  he  says,  "A  wrong  is  unredressed 
when  retribution  overtakes  its  redresser"? 

Semantic  meaning 

What  exactly  did  the  motto  on  the  coat  of  arms  mean? 

Semantic  meaning/ 
world  knowledge 
Setting 

What  was  [a]  Mason?  Was  it  a  religious  group?  or  the  work? 

The  story  probably  takes  place  during  the  Renaissance,  be- 
cause the  way  the  author  explained  what  Fortunato  was 
wearing:  he  was  wearing  tight-fitting  part-striped  dress,  and 
he  had  a  conical  cap  and  bells.  [The  student  missed  several 
details  which  imply  the  murder  occurred  in  the  late  18th  cen- 
tury.] 

Terminology 

Isn't  the  story  dramatic  as  well  as  situational  irony? 

The  range  illustrated  in  Table  2  appears  to  be  a  function  of  individual  differences; 
for  example,  metacognitive  sophistication,  reading  ability,  general  world  knowledge, 
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and  field  dependence/independence.  (For  a  detailed  discussion  of  the  role  of  individ- 
ual differences  in  journal  writing,  see  Cole,  1992.)  Through  questions  or  comments, 
students  may  try  to  (a)  clarify  facts,  and  allusions,  (b)  verify  inferences  about  world 
knowledge  or  character  motivation,  (c)  apply  literary  terminology  (e.g.,  identify  in- 
stances of  foreshadowing,  irony,  the  conflict,  the  crisis,  or  the  climax),  (d)  analyze  or 
evaluate  the  various  literary  elements  (e.g.,  setting,  motivation,  or  point  of  view),  (e) 
problem  solve  to  identify  the  theme,  or  (f)  evaluate  a  story  in  terms  of  its  subject  or 
theme,  its  plausibility,  its  artistic  merit,  etc.  Equally  interesting  is  the  fact  that 
while  every  class  of  students  as  a  whole  tends  to  have  similar  questions  and  com- 
ments, each  class  has  unique  questions  and  comments  that  reveal  mqjor  bugs  in  their 
understanding  or  astute  insights.  On  the  one  hand,  the  similarity  between  groups  of 
students  allows  the  teacher  to  utilize  a  general  instructional  design;  on  the  other 
hand,  the  uniqueness  requires  "on  the  fly"  adjustments  to  students*  needs. 


Formative  Evaluation 

Not  all  adjustments  must  occur  "on  the  fly."  Instructional  design  and  develop- 
ment can  identify  some  of  the  questions  and  comments  during  traditional  formative 
evaluation,  and  provide  scaffolding,  if  appropriate,  for  future  learners.  For  example, 
formative  evaluation  which  incorporated  student  journals  could  certainly  have  de- 
tected the  somewhat  common  misconception  that  Montresor  was  just  going  to  play  a 
joke  on  Fortunato  (UI  was  sure  Montresor  was  just  playing  a  joke  on  Fortunato")  as 
well  as  the  reason  for  this  misreading.  This  is  one  of  the  misconceptions  that  derives 
from  students'  inferring  inaccurate  meanings  of  the  word  immolation  (in  the  opening 
of  the  story),  rather  than  taking  time  to  verify  the  meaning  in  a  dictionary.  Although 
textbook  editors  usually  footnote  definitions  of  a  few  of  the  words  in  that  story  (e.g., 
pipe,  roquelaire,  flambeaux,  In  pace  requiescat),  they  never  include  immolation;  yet, 
clearly,  they  should. 

But  again  it  is  important  to  emphasize  that  formative  evaluation  does  not  stop 
when  instruction  is  implemented — it  is  an  ongoing  process  whose  purpose  is  to  im- 
prove the  product.  Because  each  group  of  students  brings  its  own  needs  to  the  learn- 
ing situation,  every  group  of  students  is  likely  to  flush  out  previously  undetected  er- 
rors or  limitations  of  the  instruction. 


Implications  for  Instruction  and  Learning 

Respecting  a  learner's  questions  and  comments  encourages  exploration;  the  stu- 
dent's questions  and  comments  can  scaffold  his/her  own  learning.  Using  a  vehicle 
such  as  journal  writing,  by  which  the  learner  can  share  questions  and  comments  in  a 
non-threatening  context,  gives  the  teacher  the  opportunity  to  (a)  monitor  the  studen- 
t's comprehension;  (b)  identify  misconceptions  which  interfere  with  learning,  or  in- 
sights; and  (c)  in  keeping  with  the  goal  of  empowering  the  learner,  provide  just 
enough  help  for  the  learner  to  complete  the  task  (e.g.,  Burton  &  Brown,  1979; 
Resnick,  1983). 

Student-generated  questions  and  comments  can  scaffold  instructional  design,  de- 
velopment, and  implementation  by: 

(a)  facilitating  teacher's  decomposing  his/her  expert  knowledge, 

(b)  helping  the  teacher  assess  the  students'  prior  knowledge  (including  misun- 
derstandings), 

(c)  providing  a  basis  for  selecting  and  sequencing  instruction  (the  teacher  is  free 
to  choose  whatever  sequence  and  method  he/she  wishes). 
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Student-generated  questions  and  comments  can  serve  as  an  instructional  tool  in 
the  following  ways: 

(a)  individualizing  instruction  by  identifying  problems  individual  students  en- 
counter in  interpreting  instructional  materials; 

(b)  providing  specific  feedback  in  a  timely  and  appropriate  manner;  for  example, 
the  teacher  can  respond  in  writing  in  journals,  tutor  students,  or  recommend 
other  remedial  or  enrichment  activities;^ 

(c)  developing  rapport  between  student  and  teacher; 

(d)  motivating  students  by  encouraging  them  to  interact  with  text  on  their  own 
levels.  ^ 

Journal  writing  can  itself  facilitate  instruction  by  providing  an  incentive  to  read 
assignments  reflectively  before  class  discussion  (e.g.,  Bauso,  1988;  Cole,  1992; 
Schwartz,  1989).  If  students  have  not  read  an  assignment,  it  is,  of  course,  impossible 
to  engage  them  in  meaningful  dialogue  in  class — identifying  and  addressing  miscon- 
ceptions and  going  beyond  the  foundation  provided  by  the  text.  Without  a  meaning- 
ful incentive  for  students  not  only  to  read  assignments  but  also  to  read  them  reflec- 
tively, the  best  laid  plans  of  instructional  designers,  developers  and  teachers  often  go 
astray.  In  my  research  (Cole,  1992),  the  brightest  student  described  himself  as  "lazy" 
and  said  he  would  not  have  thought  about  an  assignment  until  class  discussion  un- 
less he  had  been  required  to  write  journals.  He  said  he  found  journal  writing  helpful, 
and  even  enjoyed  it,  and  recommended  requiring  journal  writing  of  future  students. 

Finally,  because  reading  journals  is  so  time-consuming,  after  they  have  read 
journals  on  a  given  work  from  a  couple  classes  teachers  might  be  tempted  not  to  as- 
sign future  students  journals  on  that  work,  basing  instruction  on  the  questions  and 
comments  of  former  students.  Although  each  class  of  students  tends  to  address  the 
same  issues,  teacher!*  should  resist  such  temptations.  First,  each  class  raises  some 
unique  issues.  More  importantly,  one  of  the  major  purposes  of  journal  writing  is  to 
help  each  student  become  an  independent  learner.  Thus  instruction  must  provide  ex- 
tensive practice  for  each  student  to  develop  his/her  ability  to  identify  problems  and 
explore  solutions. 

Although  these  implications  for  instruction  and  learning  focus  on  the  teacher  and 
students,  formative  evaluation  allows  them  just  as  readily  to  translate  into  benefits 
for  designers  and  developers. 


Recommendations 

Students'  questions  and  comments  have  been  relatively  discounted  in  the  tradi- 
tional instructional  design,  development,  and  implementation  process,  yet  construe- 
tivist  paradigm  as  well  as  empirical  evidence  suggests  that  students'  questions  and 
comments  have  important  implications  for  instruction  and  learning.  While  research 
must  try  to  shed  light  on  this  issue,  I  would  like  to  offer  a  few  recommendations  for 
every  instructional  project  (see  Cole  [19921  for  additional  recommendations  for  im- 
plementing  journal  writing): 

1.   Determine  when  students'  questions  and  comments  can  be  elicited  and  addressed 
in  the  instructional  design,  development,  and  implementation  process. 

In  theory,  it  is  desirable  to  include  them  at  every  stage,  but  this  may  be  im- 
practicable. If  so,  the  layers-of-necessity  instructional  development  mcdel 
(Tessmer  &  Wedman,  1990;  Wedman  &  Tessmer,  1990)  allows  for  incorporating 

my  research  (Cole,  1992),  several  students  identified  having  their  questions  answered 
as  the  most  helpful  aspect  of  journal  writing.  The  implications  are  that  class  discussion  would 
not  have  raised  those  questions  and  that  they  would  not  have  asked  those  questions,  even 
though  the  students  seemed  to  participate  readily  in  class. 
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them  in  later  versions  of  a  project.  Implementation  should  always  provide  the 
opportunity  for  questions  and  comments. 

2.  Determine  how  students'  questions  and  comments  can  and  should  be  elicited  and 
addressed  in  the  instructional  design,  development,  and  implementation  process. 

What  medium  should  be  used  (e.g.,  class  discussion,  journals,  e-mail,  man- 
ual,...)? The  nature  of  the  domain  as  well  as  the  task  influence  this  issue.  For 
example,  if  the  task  is  procedural,  the  student  needs  an  immediate  response,  es- 
pecially if  injury  could  result;  a  journal  would  be  totally  inadequate,  if  the  do- 
main is  well-defined,  a  manual  or  on-line  help  system  might  be  appropriate.  On 
the  other  hand,  if  the  domain  is  ill-defined  and/or  the  task  requires  reflection,  a 
journal  or  some  type,  of  collaborative  learning  (e.g.,  class  discussion  ar  electronic 
bulletin  board)  would  be  appropriate. 

The  question  of  "how*  is,  of  course,  related  to  "when."  For  example,  surveys, 
interviews,  debriefings,  think-aloud  protocols,  on-line  help  systems,  and  e-mail 
might  be  appropriate  in  formative  evaluations. 

3.  Consider  whether  it  is  important  to  maintain  students'  anonymity  when  they  ask 
questions  or  comment  You  might  want  to  provide  more  than  one  vehicle  for 
questions  and  comments— one  public  (e.g.,  class  discussion,  collaborative  groups, 
e-mail),  at  least  one  private  (e.g.,  journals  shared  only  with  the  teacher,  an 
anonymous  electronic  bulletin  board). 

4.  Create  an  environment  in  which  students  feel  "safe"  in  asking  questions  and 
commenting.  Be  honest  but  tactful. 

5.  Clarify  the  purpose  of  questions  and  comments.  For  example,  if  you  use  journals, 
discuss  the  purpose  in  class.  Encourage  students  to  ask  questions  and  comment 
about  the  purpose,  the  procedure,  grading  criteria,  etc.  at  any  time  during  the 
course. 

6.  Provide  instructional  support  for  question/comment  activities  and  assignments. 
For  example,  if  you  assign  journals,  model  the  process  in  class;  provide  ungraded 
practice;  give  written  models;  provide  live  or  CAI  tutorials  on  journal  writing;  etc. 

7.  To  encourage  students  to  question  and  comment,  respond  in  a  timely  way. 

8.  Use  grading  criteria  and  feedback  which  encourage  meaningful  questions  and 
comments. 

a.  Try  to  achieve  a  golden  mean  in  grading.  Don't  grade  "tough"— it  doesn't  en- 
courage students  to  explore  (e.g.,  Cole,  1992;  Roth,  1985).  Don't  grade 
"soft"— most  students  need  the  external  motivator  (e.g.,  Holland,  1989;  Roth, 
1985),  even  when  they  find  journal  writing  helpful. 

b.  Be  careful  about  wording  of  feedback.  Try  to  encourage  students  to  extend 
their  reach.  Don't  say,  "No,  this  is  wrong."  You  might  ask  instead  how  the 
student  accounts  for  specific  details  which  conflict  with  the  student's  inter- 
pretation. When  you  discuss  journal  questions  and  comments  in  class,  don't 
identify  the  students  but  do  express  support  for  students'  trying  to  make 
sense  of  the  assignment  (even  if  they  have  a  partial  or  wrong  interpretation). 
Emphasize  that  you  don't  penalize  students  for  making  "wrong*  comments, 
being  confused,  or  asking  questions  that  reveal  they  don't  understand  the 
story. 

9.  As  a  part  of  any  mid-course  or  end-of-course  evaluations,  ask  students  to  evaluate 
whatever  question/comment  strategies  you  utilize;  for  example:  "What  did  you 
find  most  helpful  about  writing  your  journal?"  "What  did  you  find  least  helpful 
about  writing  your  journal?"  "How  helpful  did  you  find  writing  your  journal?" 
(Use  a  Likert  scale.)  How  helpful  were  job  aids  on  journal  writing,  such  as  stu- 
dent examples? 
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instruction  in  Oral  Communication 

Abstract 

This  paper  reviews  experimental  research  on  the 
pedagogical  effects  of  three  iVI  programs  in  oral 
communication.     Formative  evaluations  indicate  that 
students  enjoyed  the  level-Ill  interactive  video 
instruction  (IVI)  programs.     Results  of  an  immediate 
posttest,  delayed  posttest,  control  group  design 
indicate  that  the  IVI  program  "Coping  with  Speech 
Fright"  appears  to  be  as  effective  on  speech  fright  and 
recall  measures  as  lecture/linear  videotape  instruction 
by  two  outstanding  public  speaking  instructors. 
Furthermore,  students  in  the  IVI  condition  achieved 
significantly  higher  immediate  and  delayed  cognitive 
test  scores  and  significantly  greater  reduction  of 
speech  fright  over  a  four-week  period  than  did  students 
in  the  control  group.     Results  of  two  separate  studies 
using  an  immediate  posttest ,  control  group ,  comparison 
group  design  indicate  that  students  receiving  IVI  in 
"Constructing  Speaking  Outlines"  and  "Developing  Key 
Ideas:  The  Four  S's"  achieved  significantly  higher 
immediate  recall/application  test  scores  than  did 
students  in  the  control  group  or  the  comparison  group. 
Regression  analysis  indicated  no  significant  effects  of 
novelty,  GPA,  IVI  feedback  rating,  IVI  video  rating,  or 
nature  of  participation  (voluntary  versus  required)  on 
cognitive  test  scores  of  the  IVI  treatment  group  in 
either  study. 
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Recent  reviews  report  significant  learning 
outcomes  associated  with  IVI  (Bosco,  1986;  DeBloois, 
1988;  Gayeski  &  Williams,  1985;  Kalowski,  1987; 
Kearsley  &  Frost,  1985;  McNeil/  1989;  Smith,  1987). 
The  most  recent  review  (Fletcher,   1990) ,  which  included 
a  meta-analysis  of  47  empirical  studies  of  IVI  in 
defense  training  and  related  applications  in  industrial 
training  and  higher  education,  concluded  that  IVI  is 
more  effective  and  less  costly  across  a  variety  of 
instructional  settings  and  objectives  than  conventional 
instruction.    However,  these  findings  must  be 
interpreted  with  caution  because  methodological 
weaknesses  in  many  of  the  IVI  studies  have  been 
identified,  including:     investigator  bias,  non-random 
assignment  of  subjects,  lack  of  a  control  group, 
inadequate  definition  of  instructional  treatments, 
failure  to  measure  the  degree  to  which  the  treatments 
were  implemented  by  subjects,  artifact,  lack  of 
generalizability,  and  inadequate  sampling  (Bosco,  1986; 
Bunderson,  Baillio,  Olsen,  Lipson,  &  Fisher,  1984; 
Cushall,  Harvey,  &  Brovey,   1987;  Reeves,  1986,  1990; 
Slee,   1989;  Smith,  1987). 

Many  of  the  empirical  studies  considered  in  these 
reviews  were  conducted  in  the  hard  sciences.  Theorists 
(Biglan,  1973;  Kolb,  1981;  Kuhn,  1962;  Moses,  1990) 
have  identified  significant  differences  in  typical 
teaching  style  and  typical  learning  style  in  the  hard 
sciences  (laboratory  science  and  mathematics)  as 
compared  to  soft  skill  disciplines  (humanities  and 
social  sciences) .     Do  the  reported  pedagogical 
advantages  of  IVI  apply  to  the  soft  skill  disciplines? 
Cronin  and  Cronin  (1992)  reviewed  32  post-1984 
empirical  studies  in  soft  skill  areas  and  concluded 
that  "taken  as  a  whole,  these  studies  appear  to 
indicate  that  IVI  produces  significantly  greater 
cognitive  and  application  gains  than  conventional 
methods  of  soft  skill  instruction"   (p.  59) .  IVI 
produced  significant  cognitive  or  application  gains  in 
soft  skill  areas  such  as  reading,  management,  study 
skills,  logical  reasoning,  foreign  language,  sales 
training,  photography,  secondary  and  special  eduction, 
economics,  and  art.     Furthermore,  Cronin  and  Cronin 
(1992)  concluded  that: 

1.  IVI  with  videotape  produced  similar  learning 
outcomes  compared  to  IVI  with  videodisc. 

2.  The  significant  pedagogical  advantages  of  IVI  in 
soft  skill  areas  are  not  due  to  (a)  novelty  effects, 
(b)  superior  instructional  product  in  the  IVI  treatment 
versus  the  conventional  instruction  treatment,  or  (c) 
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additional  learning  time  possible  with  uncontrolled 
time-on-task  in  the  IVI  treatment. 

3.  IVI  may  be  more  effective  than  conventional 
instruction  in  addressing  a  wide  variety  of 
uncontrolled  student  variables.     Most  studies  reported 
a  lower  mean  standard  deviation  on  dependent  variables 
for  the  IVI  group  than  for  the  comparison  group. 

4.  Recent  IVI  research  in  soft  skill  areas  addressed 
several  of  the  methodological  criticisms  of  earlier 
research. 

All  but  two  of  these  studies  used  a 
control  or  comparison  group;  48%  used  80  or 
more  subjects;  and  61%  of  the  studies 
randomly  assigned  subjects  to  treatment 
groups.     However,  most  studies  in  this  area 
remain  open  to  the  charge  of  investigator 
bias  because  70%  were  conducted  by  the  IVI 
developer(s)  or  the  developing  organization. 
Furthermore,  none  of  these  studies  treated 
the  subjects'  actual  implementation  of  the 
interactive  video  instruction  available  as  an 
independent  or  a  dependent  variable. 

The  most  serious  methodological 
weaknesses  in  the  empirical  research  reviewed 
have  direct  relevance  to  the  necessity  of 
developing  and  testing  theories  explaining 
why  IVI  produces  significant  learning 
effects.     Researchers  failed  to  describe  the 
IVI  treatment  in  adequate  detail  in  73%  of 
the  empirical  studies,  making  interpretation 
of  findings  difficult.     Moreover,  because  no 
study  in  this  critical  synthesis  provided  a 
detailed  analysis  of  the  video  components  of 
IVI,   interpretations  of  the  unique  video 
contributions  to  learning  via  this  medium  are 
impossible.     In  addition,  although  some 
studies  measured  user  attitudes  toward  the 
medium,  no  study  that  allowed  for  learner 
control  reported  on  variations  in  actual  use 
of  the  instruct ional  program  among  users. 
Interpretations  of  the  relationship  between 
users'  attitudes  toward  IVI  and  their  actual 
instructional  use  of  the  medium  are  difficult 
absent  such  measures.     Continued  failure  to 
isolate  subject  differences  in  the  actual  use 
of  the  instructional  program  and  to  provide 
detailed  analysis  of  the  video  components  of 
IVI  make  it  difficult  to  integrate  the 
empirical  research  into  any  comprehensive 
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theory,     (Cronin  &  Cronin,  1992,  p.  69) 
Although  IVI  in  soft  skill  areas  appears  to  offer 
significant  instructional  benefits  in  cognitive 
achievement,  transfer  of  learning  to  performance, 
motivation  to  learn,  student  achievement  across 
uncontrolled  student  characteristics,  user  acceptance 
of  the  technology,  and  time  required  to  achieve  content 
mastery;  do  these  outcomes  apply  for  IVI  in  oral 
communication?    The  remainder  of  this  paper  summarizes 
experimental  studies  of  the  pedagogical  effects  of  IVI 
in  "Coping  with  Speech  Fright,"  "Constructing  Speaking 
Outlines,"  and  "Developing  Key  Ideas:  The  Four  S's." 

"COPING  WITH  SPEECH  FRIGHT"1 

Recent  research  indicated  that  approximately 
seventy  percent  of  the  population  experiences 
moderately  high  or  high  communication  apprehension  in 
public  speaking  contexts  (Richmond  &  McCroskey,   1989)  . 
High  levels  of  communication  apprehension  have  negative 
consequences  in  the  speech-making  process  such  as: 
communication  avoidance  (Beatty,   1987;  Mulac  &  Wiemann, 
1984) ;  shorter  speeches  (Beatty,  Forst,  &  Stewart, 
1986) ;  lower  self-esteem,  less  effective  public 
communication  (Daly  &  Stafford,   1984) ;  less  effective 
preparation  for  public  speaking,  increased  perceptions 
of  failure  in  speaking  (Kelly,  1984) ;  increased 
disf luencies,   less  effective  nonverbal  communication 
while  speaking  in  public  (McCroskey,  1982) ;  more 
frequent  decision-making  errors  in  constructing  a 
speech  (Beatty,   1988)  ;  and  considerable  anxiety  in 
those  with  high  levels  of  communication  apprehension  if 
they  are  forced  to  communicate  (Beatty,  1987) . 

Most  institutions  have  too  few  speech  faculty 
competent  to  provide  traditional  instruction  in  coping 
with  communication  apprehension  to  such  a  large 
population  (Ayres  &  Hopf,   1987).     Interactive  video 
instruction  (IVI)   is  capable  of  providing  effective 
training  to  large  numbers  of  students  in  a  cost- 
effective  manner. 

Research  Hypotheses 

This  investigation  involved  a  control  group  which 
received  no  formal  instruction  in  overcoming  speech 
fright,  an  IVI  group,  and  a  group  which  received 
lecture/ linear  videotape  instruction  by  outstanding 
teachers  in  cognitive  restructuring  techniques  to  cope 
with  speech  fright.     Because  the  content  of  the 
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lecture/ linear  videotape  version  of  the  lesson  was 
virtually  identical  to  the  IVI  and  because  only 
outstanding  instructors  were  used  to  present  the 
lecture/ linear  videotape  version  in  this  study,  no 
significant  differences  on  cognitive  gains  between 
treatment  groups  were  predicted.  However,  given  the 
educational  advantages  of  IVI  discussed  previously: 
HI:     Students  using  IVI  will  achieve 
significantly  higher  cognitive  test  scores 
immediately  after  treatment  than  will 
students  in  the  control  group. 
H2:     Students  using  IVI  will  achieve 
significantly  higher  cognitive  test  scores 
four  weeks  after  treatment  than  will  students 
in  the  control  group. 

After  receiving  IVI,  students  in  all  conditions 
presented  at  least  one  graded  speech  before  completing 
the  delayed-test  instruments.     Because  students  trained 
over  a  substantial  time  period  in  cognitive 
restructuring  techniques  and  given  opportunities  to 
practice  them  showed  significant  reductions  in 
communication  apprehension  (Connell  &  Borden,  1987; 
Harris,  1980),  we  expected: 

H3:     Students  using  IVI  will  achieve 
significantly  greater  reduction  in  speech 
fright  over  a  four-week  period  than  will 
students  in  the  control  group. 

Method 

Subjects.     After  adjusting  for  student  absences 
and  failures  to  follow  directions,  138  college  students 
enrolled  in  introductory  public  speaking  classes  at  a 
middle-sized,  comprehensive  university  in  the  southeast 
region  served  as  subjects.     Male  subjects  comprised  48% 
of  the  sample  and  female  subjects  made  up  52%. 
Students  received  no  extra  course  credit  for 
participation  in  the  study. 

Instructional  Materials.     Two  parallel  forms  of 
the  lesson  were  developed,  a  lecture/linear  videotape 
and  a  level-Ill  interactive  videotape  version.  The 
multimedia  IVI  module  incorporates  a 
tutorial/simulation  approach.     It  includes  carefully 
designed  orienting  activities,  questions,  feedback,  and 
review  options  to  promote  understanding.     A  user- 
friendly  design  facilitates  student  use  by  explaining 
the  nature  of  IVI,  instructing  students  in  program  use, 
and  repeating  specific  instructions  for  use  of  each 
screen  requiring  student  reaction. 

The  multimedia  approach  promotes  interest  and 
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understanding  through  humorous  graphics,  visual  memory 
cues,  and  workbook  exercises  designed  to  enable 
students  to  apply  IVI  learning  to  their  specific 
problems  of  speech  fright*     Video  adapted  to  the  target 
audience  includes  examples  of  speakers  experiencing 
speech  fright,  testimonies  of  those  who  suffer  from 
speech  fright,  examples  of  students'  negative  self- 
evaluation,  trainer-led  exercises  and  simulations  for 
developing  positive  self-evaluation,  and  a  brief 
analysis  of  other  techniques  for  coping  v/ith  speech 
fright. 

The  IVI  program  provides  a  carefully  constructed 
combination  of  learner-control  of  major  topics  to  be 
explored  (i.e.,  the  nature  of  speech  fright,  how  to 
overcome  speech  fright,  and  nonverbal  indicators  of 
speech  fright)  with  program-control  of  IVI  within  each 
major  topic.     This  allows  learners  to  focus  on  topics 
that  interest  them  and  ensures  that  they  will  follow 
the  lesson  sequence  developed  by  experts  in 
instructional  design  for  each  topic  selected  (Canelos, 
Baker,  Taylor,  Belland,  &  Dwyer,  1986) . 

Procedures  and  Design.     An  immediate  posttest, 
delayed  posttest,  control  group  design  was  used  for 
this  study.     The  independent  variable  consisted  of  the 
videotape-based  instructional  materials  with  two  levels 
(interactive  videotape  vs.  lecture/linear  videotape). 
The  dependent  variables  included  immediate  and  delayed 
cognitive  test  scores,  immediate  and  delayed  scores  on 
the  public  speaking  section  of  the  Communication 
Apprehension  in  Generalized  Contexts  instrument,  and 
immediate  responses  on  the  formative  evaluations  by  the 
treatment  groups. 

Students  in  intact  public  speaking  classes,  most 
of  whom  had  not  received  any  instruction  in  reducing 
speech  fright,  served  as  the  control  group.     In  an 
attempt  to  avoid  a  Hawthorne-type  effect,  students  in 
the  control  group  received  a  45-minute  placebo  lecture 
on  public  speaking  techniques  unrelated  to  coping  with 
speech  fright. 

Students  in  the  treatment  groups,  most  of  whom  had 
not  received  any  instruction  in  reducing  speech  fright, 
were  randomly  assigned  to  IVI  or  lecture/ linear  video 
(LLV)  treatments.     Treatments  were  conducted  during  the 
second  week  of  the  Spring  1991  semester.     The  45-minute 
LLV  treatment  was  presented  in  three  classes  by  the 
assigned  instructor  to  groups  of  eight  to  ten  students 
in  a  classroom  equipped  for  videotape  playback.  Both 
instructors  received  outstanding  teaching  ratings  from 
the  department  chair  and  averaged  4.5  out  of  5  on  their 
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roost  recent  student  evaluations.     The  instructional 
content  of  the  LLV  was  virtually  identical  to  the  IVI 
version  and  included  almost  all  the  video  included  in 
the  IVI  condition.     However,  this  video  was  presented 
as  linear  video. 

IVI  users  received  an  average  of  46  minutes  of 
individual  instruction  in  a  small  private  laboratory 
room.     This  condition  included  the  opportunity  to 
complete  workbook  exercises  but  did  not  require 
completion  of  these  exercises,  provided  no  opportunity 
for  class  discussion,  and  did  not  require  students  to 
be  exposed  to  all  the  content  instruction. 

Students  completed  a  ten-item  test  of  cognitive 
recall  (test-retest  reliability  =  .693)  and  the  CAGC-PS 
(test-retest  reliability  =  .702)   immediately  after  they 
received  the  instructional  material.     They  completed 
Form  B  of  the  recall  test  and  the  CAGC-PS  (Richmond  & 
McCroskey,  1989)  four  weeks  later. 

Results 

Cognitive  Test—Immediate.     The  results  supported 
hypothesis  l,  which  predicted  that  students  using  IVI 
would  achieve  significantly  higher  cognitive  test 
scores  immediately  after  treatment  than  would  students 
in  the  control  group.    There  was  no  significant 
difference  between  the  mean  immediate  cognitive  test 
scores  of  students  in  the  IVI  and  the  LLV  conditions. 
The  maximum  possible  score  on  the  cognitive  test  was 
10.     The  mean  score  was  8.63  for  students  in  the  IVI 
condition,  8.37  in  the  LLV  condition,  and  7.10  in  the 
control  group.     ANOVA  results  showed  significant 
differences  among  the  mean  immediate  cognitive  test 
scores  of  subjects  in  the  three  groups  (R2  =  .26,  F  [2, 
135]  m  23.92,  £  <  .0001).     The  group  differences 
accounted  for  26%  of  the  variance. 

Cognitive  Test — Delayed.     The  results  supported 
hypothesis  2,  which  predicted  that  students  using  IVI 
would  achieve  significantly  higher  cognitive  scores 
four  weeks  after  the  treatment  than  would  students  in 
the  control  group.     There  was  no  significant  difference 
between  the  mean  delayed  cognitive  test  scores  of 
students  in  the  IVI  and  LLV  conditions.     The  maximum 
possible  score  on  this  cognitive  test  was  10.     The  mean 
score  was  7.77  for  students  in  the  IVI  condition,  7.26 
in  the  LLV  condition,  and  6.22  in  the  control  group. 
ANOVA  results  showed  significant  differences  among  the 
mean  delayed  cognitive  test  scores  of  subjects  in  the 
three  groups   (£*  =  .22,  £  [2,   135]  -  18.70,  e  <  .0001). 
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The  group  differences  accounted  for  22%  of  the 
variance. 

Communication  Apprehension  Test  Scores.  The 
results  supported  hypothesis  3,  which  predicted 
significantly  greater  reduction  in  speech  fright  over  a 
four-week  period  for  students  in  the  IVI  treatment  than 
in  the  control  group.     Students  in  the  IVI  condition 
showed  a  mean  reduction  in  speech  fright  of  .866  points 
over  the  four-week  period,  while  students  in  the 
control  condition  showed  a  mean  increase  in  speech 
fright  of  .876  points.     An  independent  t-test  indicated 
that  these  mean  change  scores  were  significantly 
different  (t  [70.5]  =  -1.81,  E  <  .04).     There  were  no 
significant  differences  between  the  CAGC-PS  scores  of 
students  in  the  IVI  and  LLV  treatment  groups  on  either 
immediate  or  delayed  measures. 
Educat  iona 1  Imp 1 ica t ions 

Discussion  of  implications  relevant  to  all  three 
studies  of  IVI  in  oral  communication  will  be  presented 
in  the  final  section  of  this  paper.     Only  implications 
unique  to  each  separate  study  will  be  discussed  in 
these  subsections . 

The  finding  of  no  significant  differences  between 
the  IVI  and  LLV  treatments  may  encourage  educators  to 
investigate  innovative  applications  of  IVI  in  "Coping 
with  Speech  Fright."    The  use  of  individualized  self- 
paced  IVI  to  train  large  numbers  of  students  may  be  at 
least  as  effective,  and  perhaps  more  effective  and  less 
costly  across  a  wide  variety  of  instructional  settings 
and  objectives  than  conventional  instruction  (Fletcher, 
1990) . 

A  review  of  previous  research  comparing  IVI  and 
LLV  in  soft  skill  areas  concluded  that  "IVI  appears  to 
produce  significantly  greater  recall  or  application  of 
learning  than  linear,  video  instruction"  (Cronin  & 
Cronin,  1992,  p.  59).     Although  the  findings  of  this 
study  appear  to  be  at  variance  with  this  conclusion, 
most  previous  studies  failed  to  control  for  the  quality 
of  the  instructor  or  the  quality  of  the  instructional 
products  compared.     The  time  and  talent  devoted  to 
developing  IVI  may  produce  an  instructional  product 
superior  to  a  less  thoroughly  prepared  product  by  an 
instructor  using  LLV.     The  combination  of  outstanding 
instructors,  virtually  identical  instructional  content, 
and  the  careful  preparation  of  the  LLV  lesson  may 
explain  its  comparability  to  the  IVI  treatment  in 
significantly  enhancing  learning. 

Given  the  demonstrated  success  of  the  IVI  program 
in  cognitive  instruction,  the  CAGC-PS  scores  of 
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students  in  the  IVI  and  control  conditions  may  show 
even  more  significant  differences  if  compared  over  a 
longer  period  of  time.     Cognitive  restructuring 
training  requires  substantial  ;ime  and  speaking 
practice  for  students  to  appli  these  techniques  to 
reduce  their  specific  fears.     The  absence  of 
instructor-led  practice  in  applying  cognitive 
restructuring  techniques  to  public  speaking 
performance,  while  helpful  to  the  integrity  of  the 
research  design,  reduces  the  likelihood  of  producing 
significant  effects  on  speech  fright 

A  follow-up  analysis  of  the  results  of  this  study 
compared  cognitive  test  scores  and  the  actual  use  of 
IVI  by  subjects.     Major  variations  among  subjects' 
time-on-task  on  the  IVI  program  occurred  in  section  I 
(The  Nature  of  Speech  Fright)  and  section  II  (Cognitive 
Modification  Techniques  to  Cope  with  Speech  Fright). 
Subjects  (n  =  14)  who  spent  no  time  on  section  I  scored 
slightly  higher  on  cognitive  tests  (immediate  JJ  =  6.8, 
delayed  M  =  7,9)  than  subjects  (n  =  16)  who  spent  time 
on  that  section  (immediate  M  =  8.5,  delayed  M  =  7.7). 
Likewise,  subjects  (n  =  10)  who  spent  less  than  10 
minutes  on  section  II  scored  about  the  same  on 
cognitive  tests  (immediate  M  =  8.6,  delayed  M  =  7.7)  as 
subjects  (n  =  20)  who  spent  more  than  10  minutes  on 
that  section  (immediate  M  =  8.6,  delayed  M  =  7.8). 

Major  variations  in  time  spent  on  sections  I  and 
II  of  the  IVI  program  did  not  result  in  significant 
differences  on  immediate  or  delayed  cognitive  test 
scores.     Subjects  using  the  "Coping  with  Speech  Fright11 
IVI  appeared  to  demonstrate  appropriate  judgements 
regarding  what  material  they  needed  to  study  and  how 
much  time  they  needed  to  spend  on  "branches"  of  the  IVI 
program. 

"CONSTRUCTING  SPEAKING  OUTLINES"2 

Theoretical  Perspectives 

Research,  though  limited,  generally  supports  the 
conclusion  that  effective  organization  of  an  oral 
message  increases  recall,  attitude  change,  and  speaker 
credibility  ratings  (Daniels  &  Whitman,   1981;  McCroskey 
&  Kehrley,   1969;  Thompson,   1967;  Whitman  &  Timmis, 
1975).     The  authors  of  current  public  speaking 
textbooks  view  outlining  as  critical  to  the 
organization  process.     A  survey  of  recent  public 
speaking  texts  reveals  that  organization  is  seen  as  a 
key  to  speaker  success  and  outlines  are  seen  as  the  key 
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to  organization  (e.g.,  Sprague  &  Stuart,  1988;  Sproule, 

1991,  Verderber,  1991). 

Research  Hypotheses 

Given  the  educational  advantages  of  IVI  discussed 

previously: 

HI:     Students  using  IVI  in  "Constructing 
Speaking  Outlines"  will  achieve  significantly 
higher  cognitive  test  scores  immediately 
after  treatment  than  will  students  in  the 
control  group. 

H2:     Students  using  IVI  in  "Constructing 
Speaking  Outlines"  will  achieve  significantly 
higher  cognitive  test  scores  than  will 
students  in  the  comparison  group. 
No  previous  study  has  investigated  the  effects  of 
voluntary  versus  required  participation  in  IVI*  High 
willingness-to-communicate  subjects  are  significantly 
more  willing  to  agree  to  participate  and  significantly 
more  likely  to  participate  in  communication  research 
studies  than  low  willingness-to-communicate  subjects 
(Zakahi  &  McCroskey,   1989).     Thus,  volunteer  subjects 
in  communication  research  are  likely  tc  exhibit  higher 
willingness-to-communicate  than  students  who  are 
required  to  participate.     Furthermore,  volunteer 
subjects  are  likely  to  be  more  accepting  of  IVI  than 
non-volunteer  participants  and  thus  may  be  more 
motivated  to  learn  from  the  IVI. 

H3:     Volunteer  subjects  receiving  IVI  will 
achieve  significantly  higher  cognitive  test 
scores  immediately  after  treatment  than  will 
subjects  required  to  use  IVI. 

Several  theorists  assert  an  alternative  hypothesis 
for  the  reported  pedagogical  effects  of  IVI,  namely; 
the  novelty  effect  of  IVI  may  produce  higher  initial 
motivation  to  learn  (Clark  &  Sugrue,  1988;  Hannafin, 
1985;  Slee,  1989) .     However,  a  critical  synthesis  of 
IVI  research  in  soft  skill  areas  identified  several 
studies  that  controlled  for  novelty  effects  and 
reported  significant  pedagogical  effects  for  IVI 
(Cronin  &  Cronin,  1992).     Furthermore,  the  only 
reported  experimental  study  of  IVI  in  speech 
communication  found  no  significant  novelty  effect  for 
IVI  on  immediate  or  delayed  cognitive  test  scores. 
Neither  preference  for  IVI  over  traditional  instruction 
nor  previous  use  of  other  IVI  modules  versus  first-time 
use  showed  a  significant  association  with  cognitive 
test  scores  (Cronin,  Grice,  &  Olsen,   1992) . 

H4:     There  will  be  no  association  between 
treatment  group  subjects'  preferences  for  IVI 
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versus  traditional  instruction  and  their 
cognitive  test  scores  immediately  after 
treatment. 

H5:     There  will  be  no  association  between 
treatment  group  subjects'  previous  use  of 
other  IVI  modules  and  their  cognitive  test 
scores  immediately  after  treatment. 
The  self-paced  individualized  instruction 
available  in  IVI  may  be  more  effective  than  traditional 
instruction  for  low-prior-achievement  users.  Savenye 
and  Strand  (1989)  reported  that  the  average  student 
using  ivi  in  physical  science  instruction  demonstrated 
higher  achievement  levels  than  the  average  student 
receiving  traditional  physical  science  instruction. 
This  difference  was  even  more  pronounced  for  students 
of  lower  prior  achievement.     Furthermore,  research  on 
application-intensive  IVI  similar  to  that  used  in  this 
investigation  found  either  no  significant  difference  on 
immediate  posttests  between  high-prior  achievers  and 
low-prior  achievers  using  IVI  (Dalton,  1986;  Gray, 
1987)  or  a  significant  prior  achievement  x  treatment 
interaction  (Schaffer  &  Hannafin,   1986).     Students  with 
high  GPAs  generally  surpass  students  with  low  GPAs  on 
cognitive  tests  on  instructional  material.  However, 
IVI  may  help  close  the  cognitive-gain  gap  between 
students  with  lower  academic  achievement  and  students 
with  higher  GPAs. 

H6:     There  will  be  no  association  between 
treatment  group  subjects'  reported  grade 
point  averages  and  their  cognitive  test 
scores  immediately  after  treatment. 

Method 

Subjects.     After  adjusting  for  student  absences 
and  failures  to  follow  directions,  141  college  students 
at  a  middle-sized,  comprehensive  university  in  the 
southeast  region  served  as  subjects.     Male  subjects 
comprised  51%  of  the  sample  and  female  subjects  made  up 
49%.  * 

Instructional  Materials.     The  multimedia  approach 
promotes  interest  and  understanding  through  humorous 
graphics,  visual  memory  cues,  dual  screen  and  dual 
channel  presentations,  and  exercises  designed  to  enable 
students  to  apply  IVI  learning  to  the  construction  of 
speaking  outlines.     Examples  of  the  multimedia  design 
features  are  provided  below. 

Graphics,  voice  overs  coordinated  with  text 
summaries,  and  a  friendly  "tutor"  are  incorporated  to 
illustrate  key  principles  and  to  enhance  motivation  to 
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learn.     The  graphics  include  putting  the  tips  for 
designing  speaking  notes  on  separate  notecards  to 
highlight  the  tips  and  to  illustrate  the  use  of 
notecards  in  a  speech.     Other  graphics  serve  as 
mnemonic  devices  for  the  user.     One  graphic  uses  a 
combination  of  large  and  small  submarines  to  illustrate 
the  relationship  of  subpoints  to  main  points.     The  idea 
being  that  subpoints  are  subordinate  to  main  points 
just  as  the  small  submarines  are  arranged  under  the 
large  submarines.     A  related  graphic  identifies  types 
of  oranges  under  a  visual  of  an  orange  and  types  of 
apples  under  a  visual  of  an  apple.     Thus,  graphically 
illustrating  the  outlining  principles  that  subpoints 
must  be  related  to  main  points  and  that  main  points 
should  be  mutually  exclusive. 

The  program  also  incorporates  simultaneous  display 
on  two  screens  and  voice  overs  combined  with  text 
summaries.     Two-screen  or  two-channel  display  in  IVI 
provides  two  major  pedagogical  advantages.  First, 
these  dual  options  allow  the  user  to  become  the  editor 
of  the  program.     The  user  can  decide  whether  to  listen 
intently  to  the  voice  over,  to  listen  while  reading  the 
summary,  to  try  to  tie  in  the  graphic  with  the  text,  or 
to  integrate  all  of  these  elements.     This  editing 
option  allows  users  to  adapt  the  IVI  program  to  their 
learning  styles.     Secondly,  the  simultaneous  use  of 
several  interactive  channels  adapts  to  the  modern  TV, 
remote  control,  stimulus-load  expectations  of  users. 
The  stimulus  load  may  be  a  necessary  adaptation  to  the 
media  expectations  of  users  accustomed  to  the  quick- 
cutting  images  of  modern  music  videos,  commercials,  and 
movies,  as  well  as  routine  "channel  hopping." 

Procedures  and  Design.     An  immediate  posttest, 
control  group,  comparison  group  design  was  used  for 
this  study.     The  independent  variable  consisted  of  the 
presence  or  absence  of  videodisc-based  IVI  on 
"Constructing  Speaking  Outlines."    The  dependent 
variables  included  immediate  cognitive  test  scores  and 
treatment  group  responses  on  the  formative  evaluations. 
Analysis  of  variance  (ANOVA)  was  used  to  analyze  the 
cognitive  test  scores  among  the  treatment,  control,  and 
comparison  groups.     Following  a  significant  F,  Duncan's 
Multiple  Range  Test  was  used  to  determine  significant 
differences  among  individual  group  means.  Regression 
analysis  was  used  to  analyze  the  cognitive  test  scores 
of  the  treatment  group  with  the  nature  of  participation 
(required  versus  voluntary)  and  S's  rating  of  feedback 
in  the  IVI  program,  rating  of  the  video  in  the  IVI 
program,  reported  grade  point  average,  previous  use  of 
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IVI  on  other  topics,  computer  skills  estimate,  and 
preference  for  IVI  compared  to  traditional  classroom 
instruction  serving  as  predictor  variables. 

Students  from  non-speech  classes  in  economics, 
political  science,  health,  and  marketing  either 
volunteered  or  were  required  to  undergo  IVI.     They  were 
randomly  assigned  to  the  treatment  or  the  control 
group.     These  subjects  received  neither  instruction  in 
constructing  speaking  outlines  in  class  prior  to  the 
study  nor  extra  credit  for  their  participation  in  this 
study.     In  an  attempt  to  avoid  a  Hawthorne-type  effect, 
students  in  the  control  group  received  approximately  30 
minutes  of  placebo  IVI  in  "Developing  Key  Ideas:  The 
Four  S's."    This  instruction  provided  no  information  on 
constructing  speaking  outlines.     The  control  group 
consisted  of  25  males  and  22  females,  24  volunteered 
and  23  were  required  to  participate. 

•   The  treatment  group  received  approximately  3  5 
minutes  of  IVI  in  "Constructing  Speaking  outlines." 
The  treatment  group  consisted  of  24  males  and  2  0 
females,  24  volunteered  and  20  were  required  to 
participate.     These  ratios  were  virtually  identical  to 
those  in  the  control  group. 

The  comparison  group  consisted  of  50  students  from 
three  intact  performance  classes  in  speech  (two  in 
public  speaking  and  one  in  argumentation  and  debate) 
who  had  received  "normal"  instruction,  practice,  and 
feedback  on  constructing  speaking  outlines.  The 
comparison  group  was  included  to  compare  learning 
outcomes  from  IVI  in  constructing  speaking  outlines 
with  the  usual  instruction  on  this  topic  provided  in 
these  speech  classes. 

The  study  was  conducted  between  November  12  and 
26 ,  1991.     Participants  in  both  the  treatment  and 
control  conditions  were  shown  how  to  use  the  IVI 
program  by  a  trained  student  worker  and  were  left  alone 
to  complete  the  lesson.     Students  in  both  treatment  and 
control  groups  completed  a  sixteen-item  test  on 
constructing  speaking  outlines  (split-half  reliability 
-  .89),  a  sixteen-item  test  on  developing  key  ideas, 
and  a  formative  evaluation  of  the  instruction 
immediately  after  they  received  the  instructional 
material.     Students  in  the  comparison  group  completed 
^oth  sixteen-item  tests  during  a  regularly  scheduled 
class.     These  measures  were  randomly  ordered  and 
randomly  assigned  in  all  groups  to  control  for  an  order 
effect. 
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Results 

ANOVA  results  supported  hypothesis  1,  which 
predicted  that  students  using  IVI  on  constructing 
speaking  outlines  would  achieve  significantly  higher 
cognitive  test  scores  immediately  after  treatment  than 
would  students  in  the  control  group.     ANOVA  results 
also  supported  hypothesis  2,  which  predicted  that 
students  using  IVI  on  constructing  speaking  outlines 
would  achieve  significantly  higher  cognitive  test 
scores  than  would  students  in  the  comparison  group. 
The  maximum  possible  score  on  the  cognitive  test  was 
16.     The  mean  score  was  10.72  for  volunteer 
participants  in  the  treatment  group,  10.21  for  required 
participants  in  the  treatment  group,  7.82  for  required 
participants  in  the  control  group,  7.76  for  volunteer 
participants  in  the  control  group,  and  7.62  for 
participants  in  the  comparison  group.     These  scores 
were  significantly  different  (Rf  =  .30,  F  =  [4,  136]  = 
14.72,  r;  <  .0001).     The  group  differences  accounted  for 
30%  of  the  variance. 

Regression  analysis  did  not  support  hypothesis  3 
that  volunteer  participants  would  achieve  significantly 
higher  cognitive  test  scores  immediately  after 
treatment  than  would  subjects  required  to  participate. 
The  mean  cognitive  test  scores  in  the  treatment  group 
for  volunteers  (H  =  10.72)  and  required  participants  (M 
=  10.21)  were  not  significantly  different  (£  [2,  41]  *= 
.61,  e  <  .4382).     The  addition  of  this  predictor 
variable  to  the  regression  model  increased  the  B2  by 
only  .014   (see  Table  1). 


Insert  Table  1  about  here 


Regression  analysis  indicated  that,  as 
hypothesized  (4  and  5),  there  was  no  significant 
novelty  effect  on  cognitive  test  scores  in  the 
treatment  group.     No  significant  association  was  found 
between  immediate  cognitive  scores  in  the  IVI  treatment 
and  variations  in  subjects'  previous  use  of  other  IVI 
modules   (R2  =  .05,  F  [1,  42]  =  2.23,  £  <  .1432). 
Likewise,  there  was  no  significant  association  between 
immediate  test  scores  and  Variation?   in  preference  for 
IVI  versus  conventional  instruction  (£  [7,  36]  =  .10,  E 
<  .7582).     The  addition  of  this  predictor  variable  to 
the  regression  model  increased  the  fi2  by  only  .002  (see 
Table  1) . 
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Regression  analysis  supported  hypothesis  6  that 
there  would  ba  no  significant  association  between 
variations  in  GPA  and  cognitive  test  scores  immediately 
after  treatment.    There  was  no  significant  association 
between  variations  in  GPA  and  cognitive  test  scores  in 
the  IVI  treatment  group  (f  [4,  39]  =  .26,  e  <  .6158). 
The  addition  of  this  predictor  variable  to  the 
regression  model  increased  the  R2  by  onlv  .006  (see 
Table  1) .  J  1 

Educational  Implications 

IVI  may  enhance  the  effectiveness  of  video  by 
facilitating  active  discovery  and  application  on  the 
part  of  the  learner  (Hamilton  &  Taylor,  in  press; 
Hansen,   1989) .     Students  reported  that  the  video 
portion  of  the  IVI  was  helpful  in  promoting 
understanding  of  the  material  and  made  the  IVI  more 
enjoyable  and  more  interesting.     However,  scores  on 
this  three- item  measure  of  the  pedagogical 
effectiveness  of  the  video  portion  of  the  IVI   (U  = 
1.87,  ££)  =  .64,  Cronbach's      =  .48)  showed  no 
significant  relationship  with  students'  cognitive  test 
scores.     More  precise  descriptions  of  the  nature  and 
functions  of  the  video  portions  of  IVI,  more  reliable 
self -report  measures,  and  measurements  of  students' 
actqal  use  of  the  video  options  would  promote  theory 
building  and  theory  testing. 

Feedback  constructs  developed  in  the  communication 
discipline  may  be  helpful  in  designing  IVI  and 
explaining  empirical  results.     Positive  feedback  from  a 
highly  credible  source  and  feedback  that  is  perceived 
as  informative  (versus  controlling)  produces  increased 
internal  motivation  (Cusella,  1980).    Most  students 
reported  that  the  feedback  in  the  IVI  program  was 
believable,  informative,  and  did  not  attempt  to  control 
their  behavior  regarding  constructing  speaking 
outlines.     However,  the  reliability  of  the  three-item 
additive  index  of  students'  perceptions  of  the 
pedagogical  effectiveness  of  the  feedback  in  the  IVI 
program  was  so  low  (Jj  =  2.04,  Sfi  =  .56,  Cronbach' s  qC  = 
.12)  that  no  definite  conclusions  can  be  drawn 
regarding  contributions  of  IVI  feedback  to  learning. 
More  reliable  measures  of  the  various,  and  perhaps 
distinct,  functions  of  feedback  in  IVI;  more  precise 
descriptions  of  the  nature  and  functions  of  the 
feedback  design  in  IVI;  and  measurements  of  students' 
3Ctual  use  of  the  feedback  options  would  promote  theory 
building  and  theory  testing. 
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"DEVELOPING  KEY  IDEAS:     THE  FOUR  S's" 

This  study  was  conducted  concurrent  with  the 
"Constructing  Speaking  Outlines"  study.  The 
theoretical  perspectives  were  similar  in  exploring  the 
effects  of  organization  in  oral  communication. 
However,  this  study  focused  on  the  organization  of  key 
ideas  via  signposting,  stating,  supporting,  and 
summarizing  each  key  idea.     Except  for  substituting 
"developing  key  ideas"  for  "constructing  speaking 
outlines,"  the  research  hypotheses,  educational 
implications,  and  the  method  were  identical  to  those 
described  in  the  "Constructing  Speaking  Outline"  study 
with  three  exceptions.     (1)  The  split-half  reliability 
of  the  cognitive/application  test  in  this  study  was 
.891  and  Cronbach's  <£_  for  the  three-item  measures  of 
the  pedagogical  effectiveness  of  the  video  portion  and 
the  feedback  portion  of  the  IVI  were  .65  and  .68 
respectively.     (2)  The  instructional  material  provides 
a  tutorial  approach  in  this  multimedia  IVI  module.  It 
includes  carefully  designed  orienting  activities, 
questions,  feedback,  graphics,  video  simulations,  and 
review  options  to  promote  understanding.    A  user- 
friendly  design  facilitates  student  use  by  explaining 
the  nature  of  IVI,  instructing  students  in  program  use, 
and  repeating  specific  instructions  for  use  of  each 
screen  requiring  student  reaction.     (3)  The  subjects  in 
the  treatment  group  in  this  study  served  as  the 
subjects  in  the  control  group  in  the  "Constructing 
Speaking  Outlines"  study  and  vice  versa . 

Results 

ANOVA  results  supported  hypothesis  1,  which 
predicted  that  students  using  IVI  on  developing  key 
ideas  would  achieve  significantly  higher  cognitive  test 
scores  immediately  after  treatment  than  would  students 
in  the  control  group.     ANOVA  results  also  supported 
hypothesis  2,  which  predicted  that  students  using  IVI 
on  developing  key  ideas  would  achieve  significantly 
higher  cognitive  test  scores  than  would  students  in  the 
comparison  group.    The  maximum  possible  score  on  the 
cognitive  test  was  16.     The  mean  score  was  13.96  for 
volunteer  participants  in  the  treatment  group,  13.14 
for  required  participants  in  tljgp  treatment  group,  4.95 
for  required  participants  in  the  control  group,  4.96 
for  volunteer  participants  in  the  control  group,  and 
7.27  for  participants  in  the  comparison  group.  These 
scores  were  significantly  different  (B2  =  .60,  F  =  [4, 
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138]  =  51.75,  £  <  .0001).     The  group  differences 
accounted  for  60%  of  the  variance. 

Regression  analysis  results  did  not  support 
hypothesis  3  that  volunteer  participants  would  achieve 
significantly  higher  cognitive  test  scores  immediately 
after  treatment  than  would  subjects  required  to 
participate  (see  Table  2).     The  mean  cognitive  test 
scores  in  the  treatment  group  for  volunteers  (M  = 
13.96)  and  required  participants  (M  =  13.14)  were  not 
significantly  different  (£  [1,  43]  -  1.56,  e  <  .219). 


Insert  Table  2  about*  here 


Regression  analysis  indicated  that,  as 
hypothesized  (4  and  5) ,  there  was  no  significant 
novelty  effect  on  cognitive  test  scores  in  the 
treatment  group.     No  significant  association  was  found 
between  immediate  cognitive  scores  in  the  IVI  treatment 
and  variations  in  subjects'  previous  use  of  other  IVI 
modules  (B2  =  .07,  £  [5,   39]  =  .10,  &  <   .756).  The 
addition  of  this  predictor  variable  to  the  regression 
model  increased  the  E2  by  only  .002   (see  Table  2). 
Likewise,  there  was  no  significant  association  between 
immediate  test  scores  and  variations  in  preference  for 
IVI  versus  conventional  instruction  (£  [7,  37]  =  .00,  e 
<  .979).     The  addition  of  this  predictor  variable  to 
the  regression  model  increased  the  fi2  by  .000  (see 
Table  2) . 

Regression  analysis  results  supported  hypothesis  6 
that  there  would  be  no  significant  association  between 
variations  in  GPA  and  cognitive  test  scores  immediately 
after  treatment.     There  was  no  significant  association 
between  variations  in  GPA  and  cognitive  test  scores  in 
the  IVI  treatment  group  (£  [3,   41]  =  .47,  e  <  .495). 
The  addition  of  this  predictor  variable  to  the 
regression  model  increased  the  fi2  by  only  .011  (see 
Table  2) . 


GENERAL  EDUCATIONAL  IMPLICATIONS 


ERLC 


The  apparent  efficacy  of  IVI  in  significantly 
enhancing  subjects'  learning  of  oral  communication 
techniques  in  each  of  the  three  studies  is  consistent 
with  research  on  the  effects  of  IVI  in  related  soft 
skill  areas  (Cronin  &  Cronin,   1992).  This  finding 
appears  to  indicate  that  a  well-designed  IVI  program 
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can  prove  effective  in  training  students  in  oral 
communication. 

The  formative  evaluations  conducted  in  each  of  the 
three  studies  indicate  that  IVI  users  appeared  to  be 
highly  motivated  to  learn  from  the  medium.     Most  users 
found  the  IVI  programs  interesting  and  enjoyable  and 
wanted  to  learn  more  about  oral  communication  after 
taking  the  training.     IVI  users  felt  that  they  would  be 
capable  of  using  the  training  to  improve  their  oral 
communication  competency.     Likewise,  none  of  the  IVI 
subjects  found  it  difficult  to  use  the  programs.  This 
finding  may  reinforce  the  results  of  a  recent  study 
(Cennamo,  Savenye,  &  Smith,  1991)   in  which 
undergraduate  students  perceived  that  it  was 
significantly  easier  to  learn  from  IVI  than  from 
instructional  television  and  television. 

The  speaking  outline  and  developing  key  ideas 
studies  found  no  significant  effects  of  GPA,  IVI 
feedback  rating,  IVI  video  rating,  or  nature  of 
participation  (voluntary  versus  required)  on  cognitive 
test  scores  in  the  IVI  treatment  groups.  Furthermore, 
the  most  plausible  competing  hypotheses  did  not  appear 
to  explain  results  in  these  studies.     Time-on-task  and 
instructional  content  were  equivalent  for  the  IVI  and 
lecture/linear  video  groups  in  the  speech  fright  study 
which  reported  that  IVI  was  as  effective  as  instruction 
from  outstanding  teachers.     The  novelty  effect 
associated  with  IVI  (Clark  &  Sugrue,  1988;  Hannafin, 
1985;  Slee,   1989)  did  not  appear  to  explain  results  in 
any  of  the  three  studies. 

Perhaps  the  most  important  outcome  of  these 
studies  is  the  preliminary  empirical  documentation  of 
the  instructional  effectiveness  of  IVI  in  oral 
communication.     Prior  to  these  studies,  very  little 
research  supporting  applications  of  IVI  in  oral 
communication  was  available. 


NOTES 

1  Material  in  this  section  is  taken  from  Cronin, 
Grice,  and  Olsen  (1992). 

2  Material  in  this  section  is  taken  from  Cronin  (1992, 
November) . 
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TABLE  1 


"Constructing  Speaking  outlines" 
Treatment  Group  Increments  to  E2  and  e  for 
Each  Predictor  Variable 


Variable 

Increment 

F 

df 

SE 

b 

Previous  use  of  IVI 

.0503 

2.23 

1,  42 

.279 

.143 

Required  vs.  voluntary 

.0643 

.014 

1.41 

2,  41 

.548 

.256 

Feedback  rating 

.0755 

.011 

1.09 

3,  40 

.754 

.364 

Reported  GPA 

.0816 

.006 

.87 

4,  39 

.651 

.492 

Computer  competence 

.0880 

.006 

.73 

5,  38 

.273 

.602 

Video  rating 

.0946 

.006 

.64 

6,  37 

.683 

.693 

Preference  for  IVI 

.0970 

.002 

.55 

7,  36 

.287 

.788 
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TABLE  2 

"Developing  Key  Ideas:    The  Four  S's" 
Treatment  Group  Increments  to  g2  and  p_  for 
Each  Predictor  Variable 


Variable 

Increment 

£ 

df 

SE 

Required  vs.  voluntary 

.0349 

1.56 

1.  43 

.569 

.219 

Video  rating 

.0557 

.0208 

.93 

2.  42 

.568 

.342 

Reported  GPA 

.0665 

.0108 

.47 

3.  41 

.611 

.495 

Computer  competence 

.0716 

.0051 

.22 

4,  40 

.255 

.642 

Previous  use  of  IVI 

.0739 

.0023 

.10 

5.  39 

.233 

.756 

Feedback  rating 

.0742 

.0003 

.01 

6,  38 

.544 

.918 

Preference  for  IVI 

.0742 

.0000 

.00 

7,  37 

.350 

.979 
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Abstract  ' 

The  role  of  technology  in  two  restructuring  schools  is  investigated  in  light 
of  past,  present,  and  future  implementations.  The  schools  have  been  involved  in 
restructuring  for  two  years  and  are  collaborating  with  nearby  West  Virginia 
University  in  that  movement.  Questionnaires,  teacher  interviews,  and  classroom 
observations  were  used  to  collect  data.  Results  showed  the  most  common  former 
uses  of  technology  included  posters,  bulletin  boards,  filmstrips,  and  overhead 
transparencies;  today  .videos  and  computers  are  used  to  a  greater  extent.  Future 
plans  include  networking  computers  and  the  acquisition  of  peripherals  for  whole- 
class  viewing  of  computer  information.  Faculty  and  administrators  believe  that 
technology  is  a  vital  part  of  the  restructuring  movement,  but  it  is  only  a  tool  to  be 
used  in  the  teacher's  repertoire. 

Technology  Implementation  in  Two  Restructuring  Schools: 
Past,  Present,  and  Future 

West  Virginia  University  and  the  Claude  Worthington  Benedum  Foundation 
have  joined  forces  to  aid  the  restructuring  movement  in  six  professional 
development  schools  (PDS)  near  the  Morgantown,  West  Virginia  area.  Commonly 
known  as  the  Benedum  Project,  it  is  a  collaborative  educational  reform  effort  in 
West  Virginia  involving  over  300  public  school  teachers,  principals,  and 
superintendents,  along  with  1 15  West  Virginia  University  faculty  members.  The 
purpose  is  to  create  a  new  vision  for  schools  and  teacher  education  based  on  the 
best  research  and  practice  available.  New  ways  of  thinking  about  teaching  and 
administration  are  being  developed  to  keep  in  touch  with  the  needs  of  today's 
society.  Classroom  practices  include  innovative  ways  of  producing  students  who 
think  critically  and  creatively.  Because  technology  has  become  so  pervasive  in  our 
world,  its  use  has  become  an  important  inclusion  in  classroom  instruction.  This 
paper1  reports  the  efforts  of  two  PDS  sites,  Morgantown  High  School  and  East 
Dale  Elementary,  in  reforming  the  educational  process  with  particular  emphasis  on 
the  implementation  of  technology  to  keep  in  step  with  the  goals  of  society.  For  the 
purposes  of  this  study,  technology  is  defined  as  facilities,  equipment,  and 
instructional  resources  used  in  educational  settings. 

Review  of  Literature 

In  the  past,  technology  implementation  in  school  reforms  meant  adding  new 
technology  to  the  traditionally  structured  classroom.  At  times,  when  used  for  seat 
work,  drill-and-practice,  or  as  an  add-on,  it  did  nothing  more  than  reinforce  passive 
learning.  New  tools  and  techniques  will  be  needed  for  restructuring  to  succeed  on  a 
large  scale.  Sheingold  (1991)  feels  that  the  most  serious  challenge  for  restructuring 
efforts  will  be  addressed  by  changing  what  and  how  students  learn  in  school.  She 
continues  by  saying  it  is  unlikely  that  ambitious  goals  for  learning  and  teaching  can 
be  met  without  widespread,  creative,  and  well-integrated  uses  of  technologies  of 
many  kinds. 

If  the  purpose  of  education  is  to  prepare  us  for  life  in  the  world,  then  the 
technology  that  is  already  being  utilized  in  the  world  should  be  made  available  to  the 
students.  Because  technology  is  more  pervasive  in  the  world  outside  the 
classroom,  it  is  now  more  legitimized  for  use  in  the  school.  As  Sheingold  (1991) 
clearly  points  out,  "authentic  and  legitimate  work,  work  that  has  real  connection  to 
the  world  outside  school,  is  likely  to  be  engaging  and  memorable,  precisely  because 
it  does  matter"  (p.  19). 


The  research  reported  in  this  paper  was  supported  by  a  grant  from  the  Claude  Worthim-ton 
Benedum  Foundation. 
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Collins  (1991)  has  done  extensive  research  in  computer  use  in  the 
classroom.  He  finds  three  distinct  areas  of  classroom  computer  application:  (a) 
computers  used  as  tools  (word  processing,  programming  skills,  networking  in  lab 
settings,  spreadsheets,  and  data  base  applications);  (b)  computers  used  as  integrated 
learning  systems  (exercises,  and  students  portfolios);  and  (c)  computers  used  as 
activity/teaching  units  (simulations,  games,  and  intellectual  motivation  uses). 

When  technology  is  applied  and  implemented  as  part  of  restructuring,  it 
becomes  active,  hands-on,  student-centered  learning  (Kell,  Harvey,  &  Drexler, 
1990).  Restructuring  and  technology  function  in  tandem,  each  reinforcing  the 
other.  Technology  alone  will  not  solve  all  of  the  educational  problems;  it  must  be 
integrated  with  the  curriculum  to  improve  teaching/learning,  problem-solving  skills, 
and  enhance  critical,  creative  thinking 

Technology  in  the  classroom  is  not  without  problems.  Ray  (1991)  states 
that  the  majority  of  today's  educators  do  not  understand  the  how's,  why's,  and 
what's  of  technology.  They  recognize  a  need  to  receive  more  information  and 
training,  but  there  is  often  not  enough  time  or  financial  support  as  well  as 
instructional  materials  needed  to  do  so.  Many  students  have  to  share  computers 
and,  in  some  cases,  classes  are  held  in  labs  that  are  minimally  equipped.  When 
technology  is  utilized,  it  is  sometimes  done  so  without  a  real  purpose,  intent,  or 
goal.  It  seems  to  provide  schools  the  opportunity  to  say  they  are  technologically  in 
step.  Other  problems  include  how  to  handle  non-functional  materials  and  down- 
time while  the  hardware  is  in  repair;  the  organization  of  restructuring;  and 
technology  utilization  sometimes  so  poorly  conceived  that  directions  and  goals  are 
unclear  or  not  even  stated 

Other  barriers  to  technology  utilization  include:  (a)  lack  of  funds  to  purchase 
hardware  and  software;  (b)  time  to  prepare  the  integration  of  technology  into  the 
teaching/learning  process;  (c)  time  needed  to  finalize  goals  and  projects  for  utilizing 
technology  and  in  renovating  facilities  to  accommodate  hardware;  (d)  users 
anticipating  more  powerful  computers  and  other  hardware;  (e)  facilities 
experiencing  installation  problems;  and  (0  downtime  of  networks  (Falk,  1992). 
Summary 

Reform  efforts  are  not  new,  but  the  current  restructuring  effort  is  unique  in 
its  way  of  dealing  with  revision.  Teaching  methodologies  are  changing  because  of 
the  atmosphere  created  by  the  new  approach  to  education  as  seen  in  restructuring 
schools.  The  information  age  in  which  we  live  almost  necessitates  that  students 
know  where  and  how  to  find  what  they  need  to  know  to  succeed  in  life.  Students 
are  being  taught  to  be  critical  thinkers,  seeking  out  information  from  many  sources 
in  a  variety  of  ways.  Technology  is  one  of  those  ways  being  applied  in  the 
classroom  as  a  ramification  of  what  is  present  in  the  4<reaT  world. 

The  Benedum  Project  is  now  in  its  third  year  of  coordinating  collaborative 
efforts  between  West  Virginia  University  and  the  participating  public  schools. 
Stated  goals  of  improving  education  in  both  public  and  higher  education  practices 
are  met  in  various  ways  which  include  the  use  of  technology  in  classroom 
instruction  and  management  to  reflect  today's  societal  needs.  The  two  schools 
described  in  this  study  were  chosen  because  of  their  exemplary  use  of  technology  in 
restructuring  efforts. 

Purpose  of  the  Study 
The  purpose  of  this  study  is  to  determine  the  role  of  technology  in  the 
restructuring  movement  in  schools. 

Research  Questions 
The  specific  research  questions  of  this  investigation  are:   (1)  What 
technology  is  being  used  in  two  restructuring  schools  and  how  is  it  being 
implemented?  (2)  How  was  technology  implemented  prior  to  restructuring?  and  (3) 
What  are  the  future  plans  for  technology  acquisition  and  implementation? 
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Study  1:  East  Dale  Elementary  School 
Design  of  the  Study 

Sample 

Described  as  a  rural  school  by  virtue  of  its  population,  East  Dale 
Elementary  school  is  located  in  northern  West  Virginia.  It  is  comprised  of  644 
students  from  first  through  sixth  grade  with  38  teachers,  one  principal,  and  one  vice 
principal  collaborating  to  direct  educational  activities.  The  average  class  size  is  24 
students.  Kindergarten  students  are  located  in  another  building,  but  are  often 
transported  to  the  school  for  events. 
Method 

Information  was  gathered  through  questionnaires  completed  by  teachers  and 
administrators;  three  teacher  interviews;  and  videotaping  classroom  activities.  East 
Dale  Elementary  was  identified  as  a  PDS  site  with  a  high  use  of  technology  through 
the  recommendation  of  the  Assistant  to  the  Director  of  the  Benedum  Project  at  West 
Virginia  Lmversity.  J 

Data  from  questionnaires  were  collected  in  October,  far  enough  into  the 
school  year  for  teachers  to  be  beyond  the  beginning-of-school  activities,  yet  fully 
acquainted  with  technology  plans  for  their  classrooms.  Videotaping  and  interviews 
took  place  in  October  and  November  for  the  same  reasons. 
Procedures 

Questionnaires  relating  to  the  past,  present,  and  future  uses  of  technology 
were  distributed  to  38  teachers  and  two  administrators  at  a  faculty  meeting  by  the 
principal  with  a  request  to  return  them  in  two  weeks.  Twenty  of  these  were 
S22?^  7  5J%  retUm  ra!c-  Quantifiable  data  from  the  questionnaires  were 
entered  into  a  database  and  analyzed  for  comparative  information.  Semi-structured 
interviews  were  held  with  three  teachers  who  were  recommended  by  the  principal 
as  being  highly  involved  with  technology.  The  interviews,  approximately  20 
minutes  long,  were  audio-tape  recorded  and  transcribed.  During  one  day  of 
classroom  observations,  35-millimeter  slide  photographs  and  videotape  of  students 
Analyst  n0tCS  °f  1,1080  activities  wcre  recorded. 

To  answer  the  research  question  of  what  technology  is  being  used  in  this 
restructuring  school  and  how  it  is  being  implemented,  transcriptions  of  teacher 
nterviews  were  read  several  times;  results  of  database  information  gathered  from 
teacher  questionnaires  were  categorized  and  transformed  into  graphical 
representahons;  and  journal  notes  from  classroom  observations  were  studied 

To  answer  the  research  question  of  how  technology  was  implemented  prior 
to  restructuring,  data  from  teacher  questionnaires  were  extracted  and  graphically 
represented,  transcriptions  of  teacher  interviews  were  reviewed  and  information 
relating  to  the  use  of  technology  was  retrieved. 

To  answer  the  research  question  of  what  future  plans  are  for  technology 
acquisition  and  implementation,  teacher  questionnaires  were  scrutinized  and 
transcriptions  ot  teacher  interviews  were  studied. 

,   ,  Results 
Descriptive  Overview 

In i  1987,  East  Dale  wns  chosen  as  one  of  only  130  schools  nationwide  to  be 
named  a  National  School  of  Excellence,  an  honor  bestowed  by  the  Department  of 
Education  in  Washington,  D.  C,  after  careful  scrutiny  from  federal  observers  A 
Excd?ence    l°       dislinction  was  being  named  a  West  Virginia  School  of 

East  pale's  restructuring  movement  is  based  on  a  vision  of  education  in 
which,  collaborating  with  higher  education,  the  community,  teacher,  staff,  and 
studen  s,  the  plan  is  to  integrate  knowledge  and  critical  thinking  to  develop  the 
whole  learner.  The  school's  stated  educational  goal  is  to  motivate  students  to  value 
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education  and  become  responsible  decision  makers  and  life-long  learners.  Their 
goal  for  the  use  of  technology  is  to  expand  all  learners'  knowledge  of  current  and 
future  uses  of  technology. 

Rather  than  the  traditional  classrooms  with  rigid  walls  as  barriers,  some  of 
East  Dale's  classes  are  arranged  into  "Clusters"  in  large  areas  for  instructional 
purposes.  Cluster  100  includes  students  from  Grade  2  and  part  of  Grade  3;  Cluster 
200  has  students  in  Grade  1  and  part  of  Grade  2;  Cluster  300  students  are  from  part 
of  Grade  3  and  Grade  4;  and  Cluster  400  is  made  up  of  students  in  Grades  5  and  6. 
The  areas  are  open,  and  the  students  are  separated  into  groups,  but  they  do  contain 
the  usual  classroom  equipment  of  tables,  chairs,  teacher's  desk,  and  books.  Five 
teachers  share  all  resources  in  the  Cluster. 

The  data  collection  showed  that  East  Dale  Elementary  School  has  an 
abundance  of  technology  in  use.  Four  general  areas  of  use  surfaced  and  include: 
(1)  Clusters,  (2)  Computer  Lab,  (3)  Technology  Site,  and  (4)  Science  Lab.  Other 
departments  such  as  the  Library  and  the  Gifted  program  have  technology  in  place. 

Clusters.  Cluster  100  (Grade  2  and  part  of  Grade  3)  has  six  Apple 
computers  for  student  use  with  content-area  and  enrichment  software.  Future  plans 
include  networking  IBM  computers  with  Cluster  200  in  order  to  use  software 
installed  on  that  network.  Cluster  200  (Grade  1  and  part  of  Grade  2)  has  a  network 
of  16  IBM  computers  with  which  students  use  software  for  mathematics,  language 
arts,  spelling,  basic  skills,  and  write  their  own  stories  with  a  desktop  publishing 
program* 

In  Cluster  300  (part  of  Grade  3  and  Grade  4)  there  are  12  IBM  PCJunior 
computers.  The  software  used  is  for  word  processing,  keyboard  skills,  and  as  a 
reinforcer  for  math  instruction.  Cluster  400  students  (Grades  5  and  6)  are  sent  to 
the  Computer  Lab  in  a  separate  room  for  instruction  in  computer  skills. 

Computer  Lab.  The  Computer  Lab  at  East  Dale  has  been  in  existence  for 
several  years.  The  IBM  PCJunior  computers,  now  housed  in  Cluster  300,  were 
previously  situated  in  this  location.  This  year,  15  IBM  Model  2  computers  are  used 
by  fifth  and  sixth  graders  learning  keyboard  skills  and  word  processing.  One  day  a 
week  they  have  their  choice  of  math,  geography,  or  spelling  software  to  use.  The 
students  are  paired  for  20-minute  work  sessions  for  each  partner  in  a  40rtninute 
period  every  day. 

The  director  is  completing  her  first  year  of  teaching  in  the  computer  lab  and 
had  to  arrange  and  coordinate  all  equipment  and  activities  for  her  students.  She  also 
offers  informal  help  to  any  teacher  requesting  it.  Plans  are  in  the  making  for  fund- 
ruising  activities  to  purchase  an  overhead  projector  and  LCD  pad  for  whole-class 
viewing,  Networking  with  other  computers  in  the  school  is  a  possibility  sometime 
in  the  future. 

Technology  Demonstration  Site.  Fifth  and  sixth  grade  students  are 
exposed  to  language  arts  in  a  technology-rich  multimedia  environment.  Installed 
only  last  year  and  funded  though  a  West  Virginia  Department  of  Education 
Technology  Demonstration  Sites  Grant,  the  glass-enclosed  room  contains  a 
Macintosh  computer;  large-screen  television/monitor;  overhead  projector,  LCD  pad 
which  can  be  used  to  project  the  computer  image  to  the  entire  class;  CD-ROM  drive; 
laser  disc  player;  laser  printer;  scanner;  and  a  Zap  Shot  camera  which  allows 
pictures  taken  to  be  shown  on  the  television/monitor.  Various  software,  laser 
discs,  and  compact  discs  (CDs)  are  used  every  day  for  instructional  purposes. 

The  director  of  the  site,  who  is  the  language  arts  teacher,  has  a  total  of  168 
students  daily  in  scheduled  40-minute  periods.  In  an  interesting  use  of  technology, 
the  students  are  videotaped  while  doing  oral  presentations  of  their  written  projects. 
Software  used  for  written  expression  includes  word  processing  and  desktop 
publishing  programs.  Primarily  used  for  enhancement  activities,  the  site,  according 
to  the  director,  meets  the  needs  of  many  of  her  students  because  of  the  interactions, 
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motion,  and  lack  of  intimidation  of  the  technology.  During  one  classroom 
observation,  a  laser  disc  tutorial  of  French  vocabulary  was  being  used  and  held  the 
attention  of  all  students  as  they  repeated  aloud  what  they  saw  on  the  screen. 

Students  in  Kindergarten  through  Grade  4  are  also  exposed  to  multimedia 
when  scheduling  of  the  site  permits.  The  director  is  instructing  teachers  in  general 
use  of  the  equipment  and  reports  that  they  are  beginning  to  feel  capable  of 
controlling  it.  She  also  tries  to  integrate  in  her  teaching  any  relevant  software  that 
the  students  may  be  studying  in  other  subjects.  For  example,  laser  discs  dealing 
with  the  United  States  Presidents  and  First  Ladies  are  used  in  a  cross-curricular 
fashion  with  both  social  studies  and  language  emphasis. 

Before  the  installation  of  this  multimedia  site,  language  arts  for  fifth  and 
sixth  graders  was  taught  in  the  Clusters  with  traditional  paper-and-pencil  activities. 
Futare  plans  for  including  the  present  computer  on  a  network  are  under  discussion, 
and  the  purchase  of  software  for  other  content  areas  will  be  considered  as  the  site 
continues  to  develop. 

Science  Lab.  In  prior  years,  science  teachers  at  East  Dale  Elementary 
were  confined  to  teaching  in  their  own  classrooms  and  areas;  but,  with  the  inception 
of  the  Science  Lab  last  year,  their  students  are  now  exposed  to  a  variety  of 
technological  equipment  that  piques  the  interest  of  students  and  teachers  alike.  A 
series  of  math  and  science  software  is  used  along  with  word  processing,  but  the  six 
Apple  Ilgs  computers  were  purchased' specifically  for  use  with  the  LEGO  Robotics 
program  which  students  use  to  control  features  and  actions  of  various  projects  such 
as  carousels,  optical  scanners,  cars,  and  machines.  A  climatanum  affords 
experiments  with  different  wave  lengths  of  light,  temperature,  humidity,  and  acid 
rain  while  learning  about  plant  growth.  Other  science  equipment  used  in  the  lab 
include  microscopes,  fresh-water  marine  aquarium,  and  embossed  desk-tops  on 
which  measurement  devices,  graphs,  and  data  recorders  are  located. 

The  lab  is  used  primarily  for  fifth  and  sixth  grade;  but,  as  early  as 
Kindergarten,  students  are  exposed  to  the  LEGO  construction  principles  of  levers, 
simple  machines,  and  physical  laws.  The  director  of  the  Science  Lab  has  worked 
collaboratively  with  other  teachers  in  developing  the  curriculum  and  corresponding 
activities  for  lab  use.  He  has  spent  time  training  them  to  use  the  technology  and  is 
still  in  the  process  of  doing  so.  The  curriculum  stresses  technology  for 
communication,  manufacturing,  construction,  and  business. 

Students  from  surrounding  schools  sojourn  to  the  Science  Lab  in  scheduled 
visits.  Fourteen  schools  send  students  in  Grades  3  through  6  for  a  monthly 
exposure  of  three  hours.  Early  next  year,  the  director  will  be  supervising  after 
school  and  Saturday  programs  in  the  lab  for  nearby  communities. 

A  Macintosh  LC  computer  has  been  purchased  with  plans  for  using  it  as  a 
file  server  to  network  with  other  computers  in  the  building.  Due  to  the  recent 
origins  of  the  lab,  the  director  is  working  on  teacher  training  to  aid  in  implementing 
the  present  technology  rather  than  acquiring  more  hardware  or  software.  He  feels 
that  . .  .all  too  often  technology  and  new  things  in  the  classroom  get  put  aside  for 
lack  of  an  understanding  by  the  teachers."  With  that  in  mind,  he  is  trying  to  "settle 
in  and  make  use  of  what  we  have." 

Other  Areas.  Technology  appears  in  many  areas  of  the  school  and  it  is 
not  unusual  to  find  a  computer  sitting  by  a  table  in  an  out  of  the  way  place  such  in 
the  remedial  reading  teacher's  space  located  adjacent  to  the  Library.  The  Library 
has  two  IBM  computers  for  management  of  resources  and  student  use  of  Collier's 
Student  Encyclopedia.  There  are  two  large  screen  monitors  for  whole  class 
viewing  of  videos  and  the  school's  gifted  students  have  exposure  to  computers  in 
their  program. 
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Summary  of  Results 

Past  Implementation.  Generally  speaking,  past  uses  of  technology 
included  items  such  as  posters,  bulletin  boards,  silent  or  sound  filmstrips,  video 
tapes,  overhead  transparencies,  35  millimeter  slides,  and  limited  use  of  instructional 
television  broadcasts.  Single  computers  are  now  available  in  each  cluster  for 
teacher  use  along  with  networked  and  stand-alone  computers  for  student  use  where, 
in  past  years,  none  existed  (See  Figure  1). 
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Figure  1 

Present  and  Prior  Technology  Use  -  East  Dale 

Present  Implementation.  Students  at  East  Dale  Elementary  School  are 
exposed  to  technology  from  Kindergarten  through  sixth  grade.  General  classwork 
in  the  Clusters  is  enhanced  with  stand-alone  computers  and  those  on  networks, 
while  special  areas  such  as  the  Technology  Demonstration  Site  for  language  arts, 
the  Science  Lab,  and  the  Computer  Lab  are  available  for  instruction  in  those 
specialized  areas.  The  technology  is  up-to-date  and  being  used  regularly.  Teachers 
have  taken  the  initiative  in  learning  how  to  teach  with  technology  and  are  sharing 
their  knowledge  with  other  schools  and  students. 

Many  teachers  have  become  comfortable  with  using  various  technologies  in 
their  teaching  such  as  CD-ROM  (compact  disc-read  only  memory)  disks, 
instructional  television  broadcasts,  and  videotapes.  Some  have  reported  die  use  of 
overhead  transparencies  and  16  millimeter  films  as  new  uses  since  restructuring 
efforts  have  begun,  while  one  teacher  reported  the  use  of  a  two-way  audio 
teleconference  in  her  pedagogy. 

Today,  technology  at  East  Dale  is  used  to  the  greatest  degree  for  classroom 
instruction,  but  some  teachers  have  reported  using  it  for  managerial  purposes  and 
professional  development  as  well  (See  Figure  2).  The  computer  is  used,  at  times, 
as  a  secondary  instructional  tool  (reinforcer  for  primary  instruction)  and  as  a 
reward,  but  most  teachers  reported  using  games  and  drill-and-practice-type 
software  (See  Figure  3)  for  supplementary  instructional  activities  in  their 
educational  scheme  (Sec  Figure  4). 
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Figure  2 
Overall  Use  of  Technology 
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Figure  3 
Type  of  Software  Used 
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Figure  4 
Instructional  Use  of  Technology 

Future  Implementation.  East  Dale's  faculty  is  not  content  with  what 
they  have,  but  are  planning  future  improvements  for  their  instructional  technology. 
Networking  with  other  areas  of  the  school,  modems  for  communications  between 
schools,  more  hardware,  and  additional  software  are  on  the  agenda.  Plans  are 
directed  through  the  Technology  Committee  whose  membership  is  voluntary  and 
represents  various  categories  of  faculty  and  administration. 

A  troublesome  concern  of  the  presence  of  abundant  technology  was  voiced 
by  more  than  one  faculty  member.  There  is  a  fear  that  when  sixth  grade  students 
leave  East  Dale,  and  enter  a  school  with  less  to  offer  in  technology-related 
instruction,  they  will  not  be  as  motivated  to  learn  as  they  are  now.  The  Science  Lab 
director  quoted  a  substitute  teacher  as  saying  that  when  she  is  at  other  schools 
teaching  she  often  finds  herself  thinking  . .  if  I  only  had  East  Dale's  equipment,  I 
could  shown  them  this  [material]  so  much  more  clearly."  While  this  is  a  valid 
concern,  it  is  not  one  that  should  deter  any  use  of  technology,  rather  it  is  hoped  that 
the  secondary  schools  will  see  the  value  of  technology  and  implement  it  in  their 
various  curricula. 

Study  2:   Morgantown  High  School 
Design  of  the  Study 

Sample 

Approximately  1,381  students  in  Grades  10  through  12  attend  Morgantown 
High  School  in  Morgantown,  West  Virginia,  located  near  the  campus  of  West 
Virginia  University.  There  are  80  faculty  members,  a  principal,  and  an  assistant 
principal  in  this  large  secondary  school  that  has  an  average  class  size  of  21.5. 
Method 

Morgantown  High  School  was  recommended  by  the  Assistant  to  the 
Director  of  the  Benedum  Project  at  West  Virginia  University  as  a  school  with  a  high 
use  of  technology.  Data  from  teacher  and  administrator  questionnaires  (same  as 
Study  1)  were  collected  during  the  same  time  frame  as  Study  1  (October, 
November).  Four  teacher  interviews  were  conducted  and  videotapes  of  classroom 
use  of  technology  were  taken. 
Procedures 

Teacher  questionnaires  were  distributed  to  the  faculty  of  Morgantown  High 
by  the  principal  at  a  faculty  meeting  with  a  request  to  complete  them  in  two  weeks. 
Fifty-two  teacher  and  one  administrator  questionnaires  relating  to  the  present,  past, 
and  future  uses  of  technology  were  returned  from  a  pool  of  81  for  a  return  rate  of 
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64%.  The  quantifiable  data  collected  were  entered  into  a  database  for  comparison  '  ° 
purposes.  r 

Three  teachers  were  interviewed  and  audio-taped,  while"  one  phone 
interview  was  conducted  with  a  fourth  teacher.  The  interviews  were  semi- 
structured  and  the  teachers  were  chosen  upon  recommendation  of  the  principal  and 
as  a  result  of  a  request  for  volunteers  on  the  teacher's  questionnaire.  Classroom 
observations  were  accompanied  by  taking  still  pictures  and  videotape  of  the 
students  while  using  various  forms  of  technology.  Journal  notes  were  taken  during 
the  observations.  e 
Analysis 

To  answer  the  research  question  of  what  technology  is  being  used  in  this 
restructuring  school  and  how  it  is  being  implemented,  the  transcriptions  of  teacher 
interviews  were  read  several  times;  the  results  of  database  information  gathered 
from  teacher  questionnaires  were  categorized  and  represented  graphically;  and  the 
journal  notes  from  classroom  observations  were  studied. 

To  answer  the  research  question  of  how  technology  was  implemented  prior 
to  restructuring,  data  from  teacher  questionnaires  were  extracted  and  transformed 
into  graphical  representations;  transcriptions  of  teacher  interviews  were  reviewed 
and  information  relating  to  the  use  of  technology  was  extracted. 

To  answer  the  research  question  of  what  future  plans  are  for  technology 
acquisition  and  implementation,  teacher  questionnaires  and  transcriptions  of  teacher 
interviews  were  scrutinized. 

„      .   .  Results 
Descriptive  Overview 

u-  h  •  SWy  Stated'  vLision  of  thc  res*ucturing  movement  at  Morgantown 
High  is:  Morgantown  High  School:  A  School  for  the  21st  Century."  The  belief 
n^nolr *.? e4,raPlduchFf,csfthatare  taking  place  in  the  world,  students  must  be 
prepared  in  the  new  basics"  of  problem  solving,  decision  making,  critical  analysis, 
reasoning  and  thinking  skills,  computer  usage,  and  communications  in  addition  to 
the  basic  knowledge  required  for  usage  of  those  skills. 

Technology  at  Morgantown  High  is  viewed  generally  as  a  means  to  an  end. 
not  an  end  in  itself.  There  are  individual  teachers  that  use  up-to-date  technology  in 
their  classrooms,  but  three  computer  labs  form  the  basis  of  technology 
implementation  at  the  school.  The  long-range  goal  is  to  completely  computerize  the 
entire  school  and  to  network  all  classrooms  together  with  six  computer  stations  in 
each  classroom  for  student  use  and  one  for  teacher  use.  A  Technology  Committee 
comprised  of  faculty  members  and  administration,  meets  regularly  to  coordinate  the 
use  of  and  discuss  tue  need  for  additional  technology  at  the  sciiool 

„  Most  of  the  technological  use  at  Morgantown  High  is  centered  in  the  three 

computer  labs  which  are  categorized  by  departmental  use.  These  include  the:  (1) 
Writing  Lab,  (2)  Business  Lab,  and  (3)  Math/Computer  Science  Lab  The 
computers  in  these  three  facilities  are  networked  together  so  that  all  users  can  access 
any  software  from  any  lab.  Other  areas  of  use  include  the  journalism  department, 
any  daman     SSr00m'  Library'  and  the  availability  of  computers  on  carts  for 

u  ^riiing  f ab*'  ?,rior  t0  **  incePtion  of  the  Writing  Lab,  one  English 
teacher  had  an  Apple  II  computer  purchased  with  a  grant  for  use  in  remedial 
instruction.  After  becoming  familiar  with  the  benefits  of  technology,  she 
transported  students  to  West  Virginia  University's  Human  Resources  and 
Education  Computer  Lab  for  instruction  after  school.  She  was  gratified  to  notice 
that  even  students  who  did  not  like  "regular"  school  would  be  present  for  this  type 
of  nitelage.  Soon  after  seeing  the  advantageous  effects  of  computer  use,  the  Boanl 
of  Education  funded  the  present  English  Writing  Lab.  In  place  for  five  years,  the 
lab  is  designated  for  all  English  teachers'  use,  although  health  and  guidance 
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teachers  also  schedule  time  for  their  classes.  Various  health  and  guidance  software 
is  placed  on  the  network  server  for  use  by  those  classes. 

When  the  lab  was  originally  opened,  one  of  the  English  teachers  reported 
that  some  of  the  math  and  science  teachers  voiced  doubt  that  another  department 
could  make  use  of  a  computer  lab.  Staff  development  time  was  used  to  train  the 
teachers  in  the  use  of  computers  in  writing  instruction  with  the  result  that  now  most 
classes  use  the  lab  profusely  and  as  one  teacher  said  ". . .  our  students  are  profiting 
greatly  (from  using  the  computer].1'  Word  processing  software  along  with 
composing  process  software  such  as  Writer's  Helper  and  SEEN  are  used  to  help 
students  learn  the  techniques  of  writing  as  a  process  and  how  to  write  critical 
analyses.  The  25  IBM  computers  and  two  printers  are  starting  to  show  their  age 
from  heavy  use,  and  the  teachers  who  use  the  lab  would  like  to  see  new  hardware, 
such  as  color  monitors  and  dual  disk  drive-computers,  purchased  in  the  future. 

Business  Lab.  Word  processing,  integrated  programs^  and  typing  tutors 
are  examples  of  the  software  used  in  the  Business  Lab  which  contains  25  IBM 
networked  computers  and  two  punters.  Business  students  are  the  primary  users  of 
the  lab  and  have  help  available  from  lab  assistants  chosen  by  the  teacher.  The 
students  have  been  quoted  by  one  of  their  teachers  as  saying  that  they  . .  learn 
more  by  being  an  assistant  than  by  being  just  a  student0 

Multimedia  technology  is  also  offered  in  this  lab  using  LinkWay  software. 
The  program  allows  users  to  create  folders  of  their  own  information.  It  is  an 
elective  class  in  which  students  complete  tutorials  on  the  use  of  the  software  and 
projects  such  as  autobiographies;  at  the  end  of  the  semester,  they  will  contract  with 
a  teacher  from  another  subject  for  future  assignments  in  that  class  to  be  done  with 
multimedia.  In  a  related  use  of  multimedia,  a  laser  disc  player  is  connected  to  a 
monitor  and  is  being  tested  for  a  publishing  company  with  provided  software. 

One  of  the  teachers  stays  after  school  one  day  a  week  and  one  lunch  hour  to 
permit  students  to  use  the  facilities  beyond  their  normal  class  time.  This  is  typical 
of  the  attitude  of  the  teachers  found  at  Morgantown  High.  They  are  quite  willing  to 
cooperate  and  share  their  knowledge  of  technology  with  others,  even  on  their  own 
time.  Often  at  school  from  7:00  A.  M.  until  5:00  or  6:00  P.  M.,  some  faculty 
members  use  after-school  hours  to  learn  how  to  use  and  implement  the  latest 
developments  in  teaching  with  technology. 

Math/Computer  Science  Lab.  Subjects  such  as  Computer  Concepts, 
Computer  Science,  Transitional  Math,  Integrated  Computer  Applications,  Pre- 
Calculus,  and  Trigonometry  are  all  taught  in  the  Math/Computer  Science  Lab, 
which  is  directed  by  the  Technology  Coordinator  at  Morgantown  High  School. 
She  is  responsible  for  scheduling  time  for  classes  in  all  labs.  For  one  half  of  her 
day,  the  director  teaches  Computer  Science  classes  and  the  other  half  performs 
duties  as  lab  coordinator  which,  beside  scheduling  lab  time,  include:  repairing 
equipment;  advising  and  training  teachers  in  the  use  of  hardware  and  software; 
purchasing  software;  and  planning  for  innovative  uses  of  technology  in  the  school. 
The  labs  are  heavily  used  by  certain  subject  areas  which  results  in  other  departments 
being  able  to  schedule  the  lab  only  for  occasional  use  (See  Appendix  C  for  a  sample 
weekly  schedule). 

In  this  lab,  25  IBM  computers  and  two  printers  are  engaged  every  day  using 
instructional  software  for  the  appropriate  subject  areas.  Math  teachers  use  remedial 
programs,  fraction  tutorials,  pre-calculus,  and  trigonometry  software  to  enhance 
their  teaching.  Computer  science  classes  learn  ADA  programming.  Classroom 
management  software  and  desktop  publishing  software  are  the  newest  additions  on 
the  network  and  there  are  plans  to  train  teachers  in  their  use. 

Other  Areas.  The  journalism  department  has  a  temporary  location  this 
year  while  remodeling  is  taking  place  in  their  future  lab  area  which  will  house  a 
planned  25  to  30  Macintosh  computers.   Presently  the  school  newspaper, 
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yearbook,  and  a  county  newsletter  to  parents  is  produced  on  four  Macintosh 
computers  and  a  laser  printer,  using  word  processing  and  desktop  publishing 
programs.  Many  times  the  department  is  asked  to  produce  brochures  and  programs 
for  various  functions  or  groups  involved  with  the  school.  Advanced  journalism 
students  are  involved  in  production  of  all  the  publications.  Computer  technology 
has  been  in  use  for  about  five  years  now  and  the  journalism  teacher  would  like  to 
see  a  multimedia  lab,  television  production,  and  integration  of  classes  for  the 
development  and  viewing  of  various  in-house  productions  incorporated  for  future 
use.  He  can  also  see  uses  for  having  the  new  lab  networked,  a  scanner,  and  an 
LCD  pad  for  projections  on  a  large  screen  in  his  teaching.  The  biggest  stumbling 
block  for  these  future  goals  is  financial.  Some  plans  may  have  to  be  deleted  from 
the  new  lab  because  of  lack  of  adequate  funding. 

Morgantown  High  School  was  selected  as  one  of  28  schools  in  the  United 
States  to  test  the  IBM  Ulumedia  products,  Columbus  and  Illuminated  Books  and 
Manuscripts.  Placed  in  a  Social  Studies  classroom,  the  students  use  the  station  in 
groups,  individually,  or  for  whole-class  viewing.  Research  on  student-chosen  or 
teacher-assigned  topics  is  completed  in  a  cross-c utricular  fashion;  that  is,  the 
student  can  use  the  resources  for  assignments  in  any  class.  The  Social  Studies 
teacher  has  only  recently  become  familiar  with  using  technology  in  his  classroom 
dirough  attendance  at  a  workshop  and  has  become  an  advocate;  of  its  use.  Future 
plans  call  for  a  large  screen  monitor  to  aid  in  entire-class  viewing 
r«»n~A  to  labs,  teachers  have  access  to  Apple  computers  which  are 

Placed  on  carts  and  may  be  transported  to  individual  classrooms.  The  Library  has 
IBM  computers  and  a  TANDY  machine  with  an  encyclopedia  installed  for  student 
use* 

Summary  of  Results 

in„i  A~AaSt  'mPlfmtent.at{0»-  The  most  common,  former  uses  of  technology 
included  posters,  bulletin  boards,  filmstrips,  and  overhead  transparencies  with 
L°iS^SSCruUSL0f  instru<r,tl0?a!  television  and  computers.  Since  additional 
technology  has  become  available  in  recent  years,  there  has  been  an  increase  in  the 
nltJS ^0MPeS'  stand-a£ne  computers,  teacher-station  computers,  and 
networked  (lab)  computers.  CD-ROM  technology  and  interactive  videodiscs  are 
also  reported  new  uses  of  technology,  albeit  not  to  a  great  extent  (see  Figure  5) 
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Figure  5 

Present  and  Prior  Technology  Use  -  MHS 

  Pr*s*M  Implementation.  Three  computer  labs  form  the  basis  for  most 

computer  activity  at  Morgantown  High  School  including  the:  (1)  Writing  Lab,  (2) 
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Business  Lab,  and  (3)  Math/Computer  Science  Lab.  Due  to  the  heavy  use  of  these 
labs,  there  is  not  much  available  time  for  other  subject  areas  to  become  greatly 
involved  in  technology  use.  Individual  subject  areas  such  as  health  and  guidance 
are  scheduled  for  use  in  the  labs,  but  only  on  a  limited  basis.  A  core  of  teachers  is 
learning  to  use  ultimedia  software  and  some  are  implementing  it  in  their  teaching. 
The  journalism  department  creates  a  variety  of  publications  which  are  all  done  on 
(he  computer.  The  Library  has  computers  available  for  student  use  and 
transportable  computers  placed  on  carts  can  be  used  in  classrooms. 

Most  teachers  use  technology  in  classroom  instruction,  but  a  strong  second 
use  is  found  in  classroom  management  for  record-keeping  and  a  few  are  using  it  for 
professional  development  activities  (see  Figure  6).  Survey  results  showed  that  in 
using  technology  for  instructional  use,  it  is  being  used  mainly  as  a  supplement  to 
regular  instruction,  but  many  are  using  it  as  a  primary  and  secondary  method  of 
teaching.  One  teacher  reported  the  use  of  technology  as  a  reward  (see  Figure  7). 
Drill  and  practice  software  is  the  leading  type  of  software  being  used  in  the 
classroom,  but  tutorials  have  strong  use,  while  games  and  programming  languages 
also  have  places  in  the  curricula  (see  Figure  8). 
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Figure  7 
Instructional  Use  of  Technology 
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Figure  8 
Type  of  Software  Used 

Future  Implementation.  The  Technology  Coordinator  feels  that  the 
school  is  ready  for  computers  to  be  placed  in  individual  classrooms.  Heavy "use  of 
^SS."g,  u   md  t5"1?"?"  of  the  nonnal  flow  of  classroom  instruction  through 

fft^^JET^  *"  C°Uld  *  reCtlfied  *  havin*  technol°gy  availaWe  in 
the  teacher  s  room.  In  response  to  an  in-house  survey  of  the  faculty  a  ereat 

XSZ"rlTJ^till  faV°r  °f  e^pandi"S  technology  from  theS  t0 1  e 
!S  f  One  television  will  soon  be  present  in  each  room  and  teachers 

fUtUrC  ClaSSr°°m  Which  wi»  «-  one  teacher  and  six 

School  of° FS.7TnKHigh  Wa,S  a.Warded  the  status  of  a  West  Virginia 
S?i     Exceilence-L  T1?ey  have  made  inventive  use  of  an  older  building  in 
M^IfJE"*      yh.n?,0ey  10  be  incorporated  into  the  educational  process 
Many  teachers  and  administrators  are  convinced  of  the  worth  of  using  a  variety  of 
ap  ro  h  claSSroom  and  have  implemented  them  in  unique  cross-curricular 
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Conclusions 

In  an  almost  unanimous  voice,  the  teachers  at  East  Dale  Elementary  and 
Uicrjantown  High  School  agreed  that  technology  is  a  necessary  ingredient  in 
restructu  ing  schools.  Many  of  them  declared  that,  because  of  the  abundance  of 
technolo,  ;y  in  the  "real  world,"  the  school  should  provide  students  with  the 
knowledt  J  of  and  the  ability  to  handle  what  they  will  find  in  their  everyday  lives. 
On  the  other  hand,  with  the  same  strength  of  voice,  they  stated  that  technology  is 
only  a  tool  in  their  arsenal  and  should  be  used  as  such  in  their  teaching 
methodology. 

Time  for  training  and  money  for  funding  were  problems  raised  by  the 
faculty  at  both  schools.  The  typical  teacher's  day  is  already  full  of  demands,  and 
finding  the  time  to  learn  how  to  effectively  use  technology  is  a  stumbling  block  for 
many  of  them.  Often  teachers  use  their  own  time  to  become  acquainted  with  new 
tools.  When  the  obstacle  of  training  is  overcome,  the  problem  of  financing  modern 
technology  is  another  issue  to  be  addressed.  A  little  use  of  technology  seems  to 
create  the  desire  for  more  and  improved  technology.  Money  is  a  problem  in  all  of 
society  today,  but  especially  in  schools  where  tax  dollars  must  be  spent  judiciously. 

Both  schools  have  managed  to  find  ways  to  alleviate,  to  some  extent,  the 
problems  of  time  and  money  in  implementing  technology  in  their  schools. 
Aggressive  and  inventive  applications  for  grant  funding,  along  with  County  and 
State  contributions,  are  common  ways  of  purchasing  equipment.  Teachers  have 
voluntarily  given  of  their  own'time  on  occasion  to  learn  new  technology;  while,  in 
other  instances,  release  time  has  been  funded  for  training.  Because  of  their  strong 
feelings  for  the  place  of  technology  in  education,  faculty  and  administration  have 
worked  together  to  find  solutions  for  these  problems. 

Restructuring  efforts  have  created  an  atmosphere  of  cooperative  and 
collaborative  educational  innovations.  Cross-departmental  pooling  of  resources, 
risk-taking,  and  new  attitudes  of  ownership  for  the  educational  process  have  all 
contributed  in  allowing  instructional  activities  to  take  new  form  and  shape. 
Technology  has  played  a  vital  part  in  many  of  these  activities  and  in  making  both 
East  Dale  and  Morgantown  High  worthy  of  being  exemplars  in  educational  circles. 

The  extremely  cooperative  faculties  and  administrators  at  both  schools  have 
afforded  an  enormous  amount  of  information  from  this  study.  Future  research 
implications  include  examining  what  ingredients  are  necessary  for  teachers  to  use 
technology  in  their  classroom;  advantages  and  disadvantages  of  technology  in  the 
classroom;  if  restructuring  has  caused  an  increase  in  the  use  of  technology;  and 
funding  for  technology. 
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Abstract 

Our  prior  studies  using  science-related  concepts  and  rules  have  indicated  that 
learners  spend  twice  as  much  time  studying  feedback  after  fine  discrimination  errors  than 
they  do  after  gross  errors.  likewise,  studies  by  Kulhavy  and  his  associates  suggest  that 
learners  expend  longer  feedback  study  times  after  errors  for  which  they  had  a  high 
confidence  of  response.  The  purpose  of  the  present  study  was  to  see  if  there  were  a 
relationship  between  discrimination  error  (determined  by  content  analysis  and  tryout  data) 
and  confidence  of  response  (determined  by  self-report).  Results  indicated  that,  as  in  prior 
studies,  the  relationship  between  fine  discrimination  error  and  feedback  study  time  was 
positive.  The  relationship  between  fine  discrimination  error  and  confidence  of  response, 
however,  was  negative.  Possible  explanations  for  these  results  are  discussed. 


ERLC 


326 


Error  and  Feedback 


3 

Urror  and  Feedback:  The  Relationship  Between  Content  Analysis 
and  Confidence  of  Response 

In  his  classic  review,  Feedback  and  Written  Instruction.  Kulhavy  (1977)  proposes 
a  model  of  learner  expectancy.  Kulhavy'i  model  expresses  the  relationship  between  correct 
or  wrong  answer  post-response  feedback  given  to  a  learner  and  her  self-reported 
confidence  of  response  in  making  that  reply  in  the  first  place.  High  confidence  error 
feedback  (wrong  answer  feedback  for  erroneous  responses  the  learner  had  expected  to  be 
correct)  arc  predicted  to  yield  longer  feedback  study  times  than  either  low  confidence  error 
feedback  or  feedback  after  correct  responses. 

Kulhavy's  model  adheres  to  the  first  of  Amnions'  eight  empirical  generalizations 
which  states,  "The  learner  usually  has  a  hypothesis  about  what  he  is  to  do  and  how  he  is  to 
do  it,  and  these  interact  with  knowledge  of  performance"  (1956,  p.  281).  Based  on  pilot 
data,  Kulhavy,  Yekovich,  and  Dyer  theorized  that  learners  create  a  hierarchy  of  confidence 
in  the  the  correctness  of  their  responses.  Under  these  conditions,  learners'  reactions  to 
error  range  from  surprize  when  confidence  is  high  to  acceptance  when  confidence  is  low 
(1976,  p.  522).  These  observations  were  validated  by  several  experimental  studies 
(Kulhavy  et  al,  1976, 1979;  Kulhavy,  White,  Topp,  Chan,  &  Adams,  1985;  Lhyle  & 
Kulhavy,  1987)  and  further,  endorse  the  "common  sense  criteria"  so  often  ignored  in 
experimental  research  involving  human  learning. 

From  a  differing  perspective,  we  felt  that  an  area  of  concern  in  the  work  of  Kulhavy 
and  his  associates  was  the  reliance  on  learner's  self-reports  of  the  confidence  of  their 
responses.  In  the  studies  conducted  by  Kulhavy  and  his  associates  and  the  present  study, 
which  emulated  their  procedure,  learners  stopped  after  each  response  and  rated  their 
confidence  in  each  response.  Although  the  notion  of  a  learner-constructed  response 
hierarchy  made  intuitive  sense  to  us,  we  wondered  how  often  learners  accurately  portray 
the  response  hierarchy  with  which  the  question  was  actually  answered.  Clearly,  we  felt, 
more  sophisticated  learners  with  greater  strategic  learning  ability  would  have  an  advantage 
over  those  with  less  ability.  Likewise,  older  learners  would  have  advantages  in 
understanding  their  response  hierarchies  over  very  young  learners  (Kulhavy,  Stock, 
Hancock,  Swindell,  &  Hammrich,  1990). 

Additionally,  during  instruction,  self-report  measures  are  distracting.  In  essence, 
learners  are  asked  two  questions,  one  content-related  and  one  not.  Unless,  as  is  possible, 
the  self-report  measures  were  used  as  part  of  a  game  format  (see,  for  example,  Scarth  & 
Litchfield),  their  use  in  instructional  situations  would  be  impractical. 

Work  involving  the  use  of  rational  sets  of  concepts  and  simple  rules  (Driscoll  & 
Tessmer,  1985;  Klausmeier  &  Feldman,  1975;  Markle  &  Tieman,  1970)  have  established  a 
method  by  which  errors  of  fine  and  gross  discrimination,  two  ends  of  the  error  continuum, 
may  be  predicted.  This  method  was  adapted  to  the  computer  (Dcmpscy,  1986;  Driscoll  and 
Dempsey,  1987)  and  refined  (pempsey,  Driscoll,  &  Litchfield,  in  press)  by  comparing  the 
predictions  of  fine  and  gross  discrimination  errors  (i.e.,  content  analysis)  with  actual  on- 
task  observation  of  errors  made  during  instruction.  Our  prior  experiments  using  rational 
sets  of  concepts  have  indicated  that  learners  make  a  higher  number  of  incorrect  answers  that 
are  fine  discrimination  errors  than  gross  discrimination  errors.  Consistent  with  this 
research,  we  expected  learners  to  make  more  fine  discrimination  errors  than  gross  errors  in 
the  present  study. 

An  important  indicator  of  how  engaged  learners  are  in  the  instruction  is  the  amount 
of  time  they  spend  studying  textual  feedback  given  after  incorrect  responses.  Kulhavy  and 
his  associates  (Kulhavy,  Yehovich,  &  Dyer,  1976, 1979;  Kulhavy,  White,  Topp,  Chan,  & 
Adams,  1985)  have  asserted  also  that  students9  expectancy  for  success  is  related  to  the 
amount  of  time  students  spend  studying  feedback.  Likewise,  based  on  prior  studies  with 
science  concepts  (e.g.,  Litchfield,  Driscoll,  &  Dempsey,  1990),  we  expected  that  students 
would  i|>cnd  more  time  studying  fine  discrimination  errors  (errors  associated  with  concepts 
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which  had  limilar  attributes  to  a  correctly  classified  concept).  A  student  making  a  fine 
discrimination  error,  we  posited,  would  have  a  high  expectancy  for  success.  Fine 
discrimination  errors  were,  after  all,  close-in  nonexamples  of  correct  concepts.  Making 
incorrect  responses  that  were  "almost"  correct  should  serve  to  increase  attention  and 
stimulate  curiosity.  Under  these  condition: ,  feedback  study  time  would  be  extended 

Gross  discrimination  errors,  on  the  other  hand,  were  far-out  nonexamples  and 
suggest  that  learners  have  failed  to  comprehend  the  material.  We  expected  that  learners  will 
spend  less  time  studying  corrective  feedback  for  gross  discrimination  errors.  Failing  to 
understand  a  concent  or  rule,  learners  guess  quickly  and  move  on  to  areas  they  better 
understand.  It  may  be  supposed  that  a  learner  who  makes  a  gross  discrimination  error  has 
little  expectancy  for  success  in  classifying  that  particular  concept 

Naturally,  becauie  our  work  and  that  of  Kulhavy  and  associates  made  predictions 
based  on  assumptions  of  learners'  expectancy  for  success,  we  speculated  that  these 
approaches  were  linked  in  some  way.  The  purpose  of  the  present  study,  therefore,  was  to 
sec  if  mere  were  a  relationship  between  discrimination  error  (determined  by  content 
analysis  and  tryout  data)  and  confidence  of  response  (determined  by  self-report). 

Method 

Subjects  and  Proccr%e. 

The  subjects  in  this  study  were  63  mostly  freshman  and  sophomore  university 
student!  enrolled  in  a  biology  class  for  nonmajors.  The  class,  which  fulfilled  a  basic 
studies  requirement  for  undergraduates,  had  a  traditionally  high  enrollment  and 
unsatisfactory  pass/fail  ratio.  Subjects  comprised  three  laboratory  classes  chosen  by  the 
undergraduate  Biology  coordinator  to  participate  in  a  pilot  program  which  incorporated  the 
use  of  adjunct  computer-based  instruction  (CBI). 

Prior  to  completing  the  computer-based  instructional  module,  students  read  a  12- 
page  chapter  on  the  topic  of  substance  abuse  from  a  required  text  produced  by  the  Biology 
Department  Two  hours  of  classroom  time  were  also  devoted  to  expository  information  of 
the  module  topic.  Students  received  credit  for  completing  the  CBI  module  at  their 
convenience  during  a  10-day  period.  To  complete  the  CBI  module,  students  located  an 
unoccupied  computer  terminal  at  one  of  several  public  access  locations  on  campus  and 
sign-on"  to  the  system.  After  typing  in  their  names  and  social  security  numbers,  subjects 
were  given  all  additional  instructions  by  the  computer  program. 

Materials  and  Instruments. 

The  content  of  the  instruction  were  selected  rational  sets  of  concepts  and  rules 
related  to  a  newly-introduced,  state-mandated  substance  abuse  module.  The  rational  sets  of 
interest  in  this  study  were  types  of  drugs  (defined  concepts),  the  effects  of  drugs  on  the 
nervous  system  (rules),  and  alcohol  use  and  abuse  (rules)  and  included  44  exemplars.  An 
instructional  design  strategy,  the  rational  set  generator,  was  applied  in  the  design  and 
development  of  the  instruction.  The  rational  set  generator  is  a  matrix  model  that 
incorporates  multiple  examples  of  concepts  and  rules  and  provides  for  discrimination  and 
generalization  learning.  Discrimination  here  refers  to  the  ability  to  make  distinctions 
between  examples  ™d  nonexamples  of  concepts  and  rules.  The  interrogatory  examples 
used  in  this  study  required  that  subjects  classify  particular  concepts  or  rules  after  reading 
narrative  anecdotes  containing  varying  degrees  of  concept  or  rule  attributes.  The  CBI 
rauond  set  generator  used  an  adaptive  strategy  which  branched  subjects  to  more  difficult 
examples  after  correct  classification  and  easier  examples  after  incorrectly  classifying  or 
applying  concepts  or  rules.  Items  answered  correctly  were  discarded  from  the  program. 

Fine  and  gross  discrimination  errors  were  diagnosed  using  a  two-step  approach 
First,  content  experts  predicted  the  relative  likelihood  of  making  a  discrimination  error  for 
each  nonexample  distractor  by  considering  the  content  relationships  among  concepts  or 
rules  in  a  rational  set.  Distractors  representing  closely  related  nonexouiples,  for  example, 
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were  more  difficult  to  discriminate  than  less  closely  related  nonexamples  and  would 
represent  fine  discrimination  errors.  Thus,  nonexamples  were  rank  ordered  by  their 
"rational"  content  relationships.  Second,  before  analysis  this  predictive  relationship  was 
compared  to  actual  student  responses  and,  where  necessary,  items  were  adjusted  to  reflect 
discrimination  error  trends. 

Before  subjects  received  any  feedback,  they  were  queried  about  their  confidence  of 
response  in  a  similar  manner  to  that  proposed  by  Kulhavy  et  al  (1979).  A  five  point  scale 
(1=  lowest  confidence,  5=highest  confident)  composed  of  touch  boxes,  and  a  question 
asking  the  student  how  sure  she  was  about  her  answer  appeared  at  the  bottom  of  the 
computer  screen  immediately  after  a  content  response  was  made.  After  indicating 
confidence  of  response,  subjects  received  content  response-contingent  feedback.  Simple 
confirmation  was  provided  for  correct  answers.  After  incorrect  responses,  subjects  were 
informed  of  the  correct  concept  or  rule  in  a  standard  feedback  box  which  remained  on  the 
screen  along  with  the  interrogatory  example  until  students  chose  to  touch  the  screen  or 
press  the  keyboard  to  continue  on  to  the  next  example. 

In  the  present  study  feedback  study  time  was  collected  after  incorrect  responses 
only.  Feedback  study  time  was  defined  as  the  elapsed  time  from  the  moment  when 
response-contingent  feedback  was  first  presented  on  the  computer  display  screen  until  the 
learner  pressed  the  appropriate  key  to  view  the  next  item. 

Results 

The  results  of  the  study  indicated  that,  as  may  be  expected,  students  spent  more 
feedback  study  time  and  required  more  question-based  examples  in  studying  content 
involving  rules  than  concepts.  Otherwise,  as  Table  1  indicates,  the  patterns  were  quite 
similar  across  the  three  learning  outcomes  used  in  this  study,  i.e.,  drugs,  coordinate  (or 
rationally-related)  defined  concepts;  drugs,  coordinate  rules;  and  alcohol,  successive  (or 
nonrelated)  rules. 


Insert  Table  1  about  here 


Feedback  study  time  was  directly  correlated  with  fine  discrimination  errors 
(r  =  .456)  as  shown  in  Table  2.  As  expected,  students  spent  much  more  time  studying 
feedback  after  fine  discrimination  errors  than  gross  errors. 

Suprisingly,  confidence  of  response  was  inversely  correlated  with  feedback  study 
time  (r  =  -.469)  as  well  as  fine  discrimination  error  (r  =  -.466)  and  gross  error  (r  =  -.479). 


Insert  Table  2  about  here 


Discussion 

Although  these  findings  are  far  from  conclusive,  two  possible  explanations  could 
explain  the  negative  relationship  between  fine  discrimination  errors  and  confidence  of 
response.  These  are:  (1)  inconsistencies  with  the  learners1  self  reports  of  their  confidence 
of  response,  and  (2)  the  relationship  between  high  confidence  errors  and  effort 

In  the  Kulhavy  studies  (as  well  as  the  present  study)  learners  stopped  after  each 
response  and  rated  their  confidence  of  response.  One  wonders  how  often  learners 
accurately  portray  the  response  hierarchy  with  which  the  question  was  answered  We 
would  suppose,  for  example,  that  there  would  be  a  great  difference  in  the  reliability  of  self- 
reported  confidence  measures  among  sophisticated  learners  versus  those  with  less  ability  - 
or  older  versus  younger  learners. 
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An  initial  investigation  by  Swindell,  Greenway,  and  Peterson  (1992)  upholds  our 
suppositions.  In  a  study  with  4th  and  6th  grade  students,  these  researchers  found  that  6th 
graders  were  more  reliable  in  estimating  response  confidence  than  were  4th  graders.  They 
also  found  that  the  response  patterns  of  the  6th  graders  were  similar  to  those  of  college 
students  (Kulhavy,  Stock,  Hancock,  Swindell,  &  Hammrich,  1990),  but  response  patterns 
of  the  4th  grade  students  were  distinctly  different. 

Other  researchers  have  called  into  question  the  use  of  self-reported  confidence 
measures.  For  example,  Koriat,  Lichtenstein,  &  Rschoff  (1980)  have  found  that  people  are 
often  overconfident  in  evaluating  the  correctness  of  their  knowledge.  Their  research 
supports  the  notion  that  learner's  assessment  of  confidence  is  biased  by  attempts  to  justify 
one  s  chosen  answer.  In  discussing  self-  reports,  Borg  and  Gall  (1983)  observed,  "people 
often  bias  the  information  they  offer  about  themselves,  and  sometimes  they  cannot 
accurately  recall  events  and  aspects  of  their  behavior  in  which  the  researcher  is  interested" 
(p.  465). 

In  addition,  self  report  measures  during  instruction  arc  distracting.  Essentially, 
learners  arc  asked  two  questions,  one  content  related  and  one  not.  Thus,  the  practical  value 
of  self-report  as  an  instructional  or  motivational  design  measurement  tool  is  reduced. 

Regarding  our  second  speculation,  the  findings  of  this  study,  considered  in  respect 
to  the  existing  text-based  feedback  literature,  indicate  a  more  complex  relationship  between 
the  type  of  error  made  (determined  via  content  analysis),  expectancy  (measured  by 
confidence  of  response  scales),  and  the  amount  of  effort  a  learner  makes  (as  measured  by 
feedback  study  time)  than  had  been  suspected.  This  relationship  is  illustrated  in  Figure  1 
In  addition  to  other  factors,  we  suspect  that  confidence  of  response  measures  are  greatly 
influenced  by  specific  learning  outcomes,  the  difficulty  of  material  to  be  learned,  the 
learners  prior  knowledge,  and  the  relevance  of  the  material  to  the  learner 


Insert  Figure  1  about  here 


One  practical  implication  of  the  present  study  is  for  researchers  to  explore  more 
sophisticated  systematic  explanations  for  the  use  of  corrective  feedback  in  interactive 
insmiction.  While  the s  tendency  is  to  look  for  simpler  clarifications  such  as  those  proposed 
by  Kulhavy  (1977),  the  evidence  of  this  and  certain  other  studies  suggest  the  relationship 
among  error,expectancy,  and  feedback  is  a  complex  one.  More  recent  work  among  several 
researchers  (Dempsey,  Driscoll,  &  Swindell,  in  press;  Kulhavy  &  Stock,  1989;  and 
Bangert-Drowns,  Kulik,  Kulik,  &  Morgan,  1991)  have  begun  to  address  these  concerns  at 
least  within  the  limited  area  of  text-based  feedback.  What  is  needed  are  disciplined 
explorations  of  these  and  other  models. 
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Tabic  1 

Descriptive  Statistics  for  number  correct,  attempts,  feedback  study  time,  fine  &  gross  discrimination 
errors  and  overaJl  confidence  of  response  m  =  63V 


VARIABLE*  MEAN  SD 

B-correct  7.08  .83 

C-  correct  6.20  .99 

D- correct  12.02  2.21 

B-attempts  11.79  4.95 

C-attempts  15.05  3.95 

D-attempts  20.12  2.59 

B-FB  study  time  131.46  211.72 

C-FB  study  time  176.30  209.19 

D-FB  study  time  170.73  178.37 

B-fine  errors  .73  .61 

C-fine  errors  2.11  1.47 

D-fine  errors  1.89  123 

B-gross  errors  .56  .98 

C-gross  errors  .76  1.10 

D-gross  errors  .65  1.01 


Confidence  of  Response  3.807  .396 

(all  3  matrices) 


Note:  44  items  -  from  three  instructional  matrjr?s- 
B  matrix  =  drugs  (16  items,  defined  concepts) 
C  matrix  ■  drug  rules  (12  items,  simple  rules) 
D  matrix  =  alcohol  (16  items,  simple  rules) 


Tabic  2 

Intcrcorrclarions  among  the  variables  of  feedback  study  rime,  confidence  of  response,  fine  discrimi- 
nation errors,  and  gross  discrimination  errors  ( n  =  631 


Variable  FB  Study  Time    Conf  of  Response  Fine  Errors         Gross  Errors 


FB  Study  Time 

1.00 

Conf  of  Response 

-.469* 

1.00 

Fine  Errors 

.456* 

-.466* 

1.00 

Gross  Errors 

.204 

-.479* 

.055 

1.00 

*  p  <  0.001 
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STUDENT  PERFORMANCE  AND  ATTITUDES  USING 
PEER  COLLABORATION  IN  ACCOUNTING 


ABSTRACT 

Purpose 

SliT!?  S!udyJWM  "aducted  to  investigate  the  effects  of  peer  collaboration 
and  individual  study  as  nodes  of  practice,  using  the  lesson  content  of  a  required 
introductory  accounting  course.  The  study  also  examined  the  effects  of  using 
content  organizers  on  the  practice  worksheets. 

Method 

Data  were  analyzed  using  a  2  x  2  analysis  of  variance  comparing  posttest 
performance  across  practice  modes  (peer  collaboration  and  individual)  and  content 
organizer  treatments  (organizers  versus  no  organizers).  Data  from  the  attitude 
questionnaire  were  analyzed  for  variations  in  responses  between  the  peer 
collaboration  and  individual-work  groups. 

Results 

Students  showed  a  significantly  higher  level  of  achievement  using  the  practice 

"i'fr  "l!*1'!"*1'"'  as  weH  "  h«v,n8  «8nific.ntly  more  positive  attitudes 
towards  this  method  of  classroom  practice  in  problem  solving 

ThZ?\  ^a"  m*yi£*yt  "ntr,buted  «°  «»«  Positive  results  for  peer  collaboration. 
These  Include  providing  and  explaining  the  answer,  obtaining  feedback  and 
discussion,  and  playing  an  active  role  in  learning. 
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The  study  of  accounting  lends  itself  well  to  group  review  as  part  of  the  learning 
process.  Even  the  accounting  profession  depends  on  peer  review  and  peer 
consulting  to  strengthen  the  validity  and  accuracy  of  individual  work.  In  recent 
years,  the  profession  has  been  vocal  in  its  concern  that  higher  education,  while 
providing  technically  educated  graduates,  must  also  help  develop  the  intellectual, 
interpersonal  and  communication  skills  of  accounting  majors.  Patten  and  Williams 
(1990)  summarized  the  key  issues  that  require  a  "thorough  re-examination  of  the 
total  accounting  experience,"  citing  increased  communication  and  interpersonal 
skills  as  two  key  issues. 

Cooperative  learning  among  students  is  becoming  increasingly  popular  in 
educational  settings.  Slavin  (1987)  defines  cooperative  learning  as  having  four 
components  of  l)group  task,  2)incentive,  3)motive  and  4)behavior.  Group  work  is 
deemed  to  be  cooperative  if  the  students  work  on  a  common  assignment  (group 
task);  have  an  interdependent  reason  to  work  together  (incentive);  want  to  work 
together  (motive);  and  work  as  a  team  (behavior)  (Hooper,  in  press). 

Cooperative  learning  methods  are  usually  based  on  one  of  two  major  theories, 
developmental  and  motivational.  Developmental  theory  holds  that  the  interaction 
of  the  group  itself  leads  to  improved  achievement,  whereas  motivational  theory 
says  achievement  depends  on  the  goal/reward  structure  (Slavin,  1987).  The  reward 
structure  in  developmental  theory  appears  more  intrinsic  whereas  the  reward 
structure  in  motivational  theory  is  more  extrinsic.  Most  empirical  findings  support 
the  reward  structure  of  motivational  theory,  but  Slavin  suggests  combining  and 
reconciling  both  theories  to  provide  the  strongest  results. 
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The  area  of  applied  accounting  content,  such  as  found  in  introductory  accounting 
classes,  may  provide  a  particularly  good  environment  and  educational  setting  for 
using  peer  collaboration.  Students  often  have  difficulty  in  accounting  and  are 
frequently  uncertain  about  their  answers  to  applied  problems.  Peer  collaboration 
could  provide  a  second  opinion  of  their  work,  similar  to  what  practioners  use.  It 
could  also  give  the  students  an  opportunity  to  discuss  and  elaborate  on  both  the 
accounting  content  and  the  problem  methodology. 

Peer  collaboration,  as  a  type  of  cooperative  learning,  may  provide  a  link  between 
these  two  types  of  reward  structures.  The  ability  to  explain  or  defend  an  answer  to 
a  peer  may  increase  comprehension  and  correct  misconceptions.  These  results 
could  lead  to  improved  learning  and  could  also  provide  both  intrinsic  and  extrinsic 
rewards  for  the  students. 

The  ability  to  organize  new  information  and  extract  the  key  steps  or  rules  may 
improve  performance  (Ellis,  et  al,  1986).  Tessmer,  et  al  (1990)  proposed  an 
approach  to  teaching  and  learning  concepts  that  would  include  demonstrating  to 
the  students  a  way  to  arrange  or  think  about  new  materials.  Accounting  education 
has  also  attempted  to  help  students  organize  the  concepts  into  important  ideas  or 
"rules",  to  facilitate  both  recall  and  application  of  the  materials.  Often,  these 
content  organizers  are  verbally  expounded  by  the  instructor,  but  they  may  or  may 
not  be  used  by  the  student  in  solving  the  accounting  problems. 

The  present  study  was  conducted  to  investigate  the  effects  of  peer  collaboration 
and  individual  study  as  modes  of  practice,  using  the  lesson  content  of  a  required 
introductory  accounting  course.  The  study  also  examined  the  effects  of  using 
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content  organizers  on  the  practice  worksheets. 


The  three  primary  research  questions  were: 

1.  Which  is  the  more  effective  method  of  practice.individual  practice 
or  peer  collaboration,  in  solving  accounting  problems? 

2.  Does  the  use  of  content  organizers  with  practice  worksheets 
increase  learning? 

3.  What  are  the  effects  of  peer  collaboration  and  individual  practice 
methods  on  student  attitudes? 

METHOD 

Subjects 

The  subjects  in  the  study  were  21 1  college  students  (sophomores,  juniors,  and 
seniors)  enrolled  in  the  first  semester  of  Introductory  Accounting  at  a  large 
southwestern  university.  The  Introductory  Accounting  classes  are  prerequisites  for 
entry  into  many  of  the  professional  programs  in  the  College  of  Business 
(Accounting,  Marketing,  Management,  Computer  Information  Systems,  and 
General  Business). 

Procedures 

The  study  was  conducted  using  a  2  (individual/peer  collaboration)  X  2 
(organizers/no  organizers)  design.  This  resulted  in  four  separate  treatment  groups: 
Peer  collaboration  with  content  organizers,  peer  collaboration  without  content 
organizers,  individual  practice  with  content  organizers,  and  individual  practice 
without  content  organizers.  Students  attended  their  scheduled  classes  and  each 
class  time-block  was  randomly  assigned  to  a  treatment. 
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At  the  break-out  session  before  the  study  and  on  first  day  of  the  study,  the 

students  were  given  the  following  information  by  their  instructors: 

Our  class  has  been  selected  to  participate  in  a  study  that  will  look  at  the 
differences  between  individual  and  group  practice  .  In  order  to  have 
reliable  results,  it  is  critical  for  you  to  attend  all  three  classes.  You  will 
receive  10  points  for  attending  all  three  classes.  But  it  is  an  all  or  nothing 
reward.  If  you  miss  even  one  of  the  classes,  you  cannot  receive  any  points. 
ADDITIONALLY,  the  quiz  on  Friday  will  be  worth  10  points. 

All  the  students  attended  the  same  weekly  mass  lecture  on  Wednesdays,  but  were 

assigned  to  different  break-out  sessions  that  met  on  Mondays  and  Fridays.  The 

break-out  sections  were  taught  by  graduate  teaching  assistants  and  ranged  in  size 

from  25  to  45  students  per  session.  The  study  ran  during  three  50  minute  class 

periods,  with  the  posttest  being  administered  in  the  third  class  period. 

During  each  class  the  teaching  assistant  lectured  and  used  the  worksheet  problems 
throughout  the  lecture  to  illustrate  the  concepts  being  presented,  as  well  as  to 
provide  practice  for  the  students.  The  teaching  assistant  verbally  provided  content 
organizers  as  part  of  the  class  discussion  that  preceded  practice  in  all  groups.  The 
researcher  and  teaching  assistants  had  agreed  upon  the  organizers  to  use  on  the 
worksheets,  which  were  the  Sl>me  ones  used  most  frequently  by  the  teaching 
assistants  in  their  lectures  (E.g.,"Remember  that  debits  equal  credits;  The 
accounting  equation  requires  that  assets  equal  liabilities  and  owners  equity.").  All 
students  individually  worked  the  practice  problems  before  the  teaching  assistant 
lead  the  class  in  discussing  and  answering  the  problems. 

In  the  peer  collaboration  groups,  the  individual  work  was  followed  by  comparisons 
of  answers  between  student  partners.  The  students  worked  in  self-selected  teams  of 
two.  The  teaching  assistant  gave  the  students  instructions  to  "Work  out  your  own 
solution  and  then  consult  with  your  partner".  Each  student  spent  about  a  minute 
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working  out  his  or  her  own  solution  to  each  problem  and  then  about  a  minute 
collaborating  on  the  answer.  Each  student  had  his  or  her  own  worksheet. 

In  the  individual  practice  treatments,  the  students  worked  on  their  problems  by 
themselves  for  about  two  minutes.  In  all  the  treatment  groups,  the  teaching 
assistant  lead  a  discussion  and  answer  process  of  solving  the  practice  problem  after 
the  students  had  worked  on  a  solution. 

The  content  organizers  were  listed  and  circled  in  the  margins  of  the  worksheets 
used  by  the  Content  Organizer  groups  (eg.,  DR  ■  CR;  A  ■  L  +  OE).  The  teaching 
assistant  verbally  included  the  organizers  in  both  the  lecture  and  the  discussion 
sections  for  all  the  classes.  The  written  organizers  were  not  included  on  any  of  the 
worksheets  on  the  last  day  of  treatment.  This  change  was  made  so  that  the  group 
using  the  organizers  would  also  receive  practice  without  the  organizers,  since  the 
test  did  not  include  any  prompts. 

Materials 

Materials  consisted  of  the  required  accounting  textbook  for  the  course,  overhead 
transparencies  prepared  by  the  two  teaching  assistants,  and  the  experimental 
practice  worksheets.  The  teaching  assistants  met  with  the  researcher  prior  to  the 
study  and  reviewed  the  content,  structure  and  wording  of  the  practice  worksheets. 
Each  teaching  assistant  used  overhead  transparencies  which  they  individually 
prepared  by  summarizing  key  content  points  from  the  textbook.  The  content  and 
presentation  of  the  overheads  were  comparable.  The  practice  worksheet  problems 
were  based  on  homework  problems  from  the  textbook. 
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Criterion  Measures 

Results  were  measured  by  a  40  item  posttest,  administered  on  the  last  day  of 
treatment.  The  tests  were  scored  by  the  researcher,  who  is  an  experienced 
accounting  instructor  and  a  CPA.  The  test  covered  the  content  of  the  accounting 
lessons  taught  during  the  study.  The  K-R  21  reliability  for  the  posttest  was  .89, 
indicating  high  reliability  of  measurement.  The  teaching  assistants  converted  the 
40  item  posttest  into  a  quiz  score  of  one  to  ten  points,  consistent  with  their  weekly 
pattern  of  ten-point  quizzes.  The  test  contributed  to  the  student's  grades  as  one  of 
12  quizzes  given  throughout  the  semester. 

Students  also  completed  a  12-item  Likert-  type  attitude  survey,  scored  from  1 
(Strongly  Agree)  to  5  (Strongly  Disagree).  The  survey  also  contained  two  open- 
ended  questions. 

Design  and  Data  Analysis 

The  research  design  was  a  2  X  2  posttest-only  factorial  design.  Data  were  analyzed 
using  a  2  x  2  analysis  of  variance  comparing  posttest  performance  across  practice 
modes  (peer  collaboration  and  individual)  and  content  organizer  treatments 
(organizers  versus  no  organizers).  Data  from  the  attitude  questionnaire  were 
analyzed  for  variations  in  responses  between  the  peer  collaboration  and  individual- 
work  groups. 

RESULTS 

Table  1  shows  mean  scores  were  27.3  for  peer  collaboration  and  23.2  for 
individual  practice.  The  difference  in  favor  of  peer  collaboration  was  statistically 
significant,  F  (2,  167)  «  5.12  ,  p  <.007. 
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Insert  Table  1  here. 


As  shown  in  Table  I,  mean  scores  for  the  content  organizers  and  no  content 
organizers  were  27.8  and  25.2,  respectively.  This  difference  approached,  but  did 
not  reach,  the  .05  level  of  significance,  F  (2,  167)  -  3.53,  p  <.062.  The  interaction 
was  not  significant  for  practice  mode  by  content  organizers. 

Four  significant  differences  between  practice  modes  were  found  in  student 
attitudes.  Students  in  peer  collaboration  groups  (mean  «  2.07)  responded 
significantly  more  favorably  to  the  question  "Do  you  think  it  helps  you  understand 
accounting  to  consult  with  a  partner?"  than  did  students  in  individual  treatment 
groups  (mean  «  2.68).  When  asked  "Do  you  think  it  helps  you  learn  more  if  you 
explain  your  answer  to  another  student?"  students  in  peer  collaboration  (mean  ■ 
1.88)  responded  significantly  more  favorably  than  did  students  in  individual 
treatment  groups  (mean  =  2.54).  The  reaction  by  students  in  peer  collaboration  was 
significantly  more  favorable  to  the  question  "By  using  the  practice  worksheets,  did 
you  find  it  easier  to  understand  your  TA's  lecture?"  (means  «  1.98  and  2.41, 
respectively).  However,  students  in  individual  treatment  groups  (mean  *  3.83) 
responded  with  significantly  stronger  disagreement  to  the  question  "Did  using  the 
practice  worksheets  take  away  time  that  should  have  been  used  for  your  TA's 
lecture?"  than  the  peer  collaboration  groups  (mean  «=  3.39). 

The  student  comments  on  the  two  open-ended  questions  yielded  some  common 
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themes.  The  first  question  asked  the  students  "What  three  changes  would  you  make 
to  Introductory  Accounting  to  make  it  easier  to  understand?".  The  most  common 
answers  were  1)  More  time  in  class  ,  2)  Less  content  coverage  ("goes  too  fast"  or 
"slow  it  down"),  and  3)  Cover  materials/concepts  before  homework  is  assigned  or 
due.  The  second  question  asked  for  student  comments  and  reactions  to  the 
research  study.  Many  students  commented  that  the  research  was  a  "good  idea"  and 
"talking  about  it  made  class  more  interesting".  Students  remarked  that  the  — 
worksheets  were  helpful,  as  was  working  with  a  "partner",  because  "Two  heads  are 
better  than  one"  and  "It  helps  to  talk  about  it  and  correct  your  own  mistakes." 

At  the  end  of  class  on  the  last  day  of  treatment,  students  were  invited  to  attend  a 
de-briefing  session  to  discuss  the  study.  A  total  of  40  students  attended  the  session 
on  their  own  time  and  participated  very  actively  in  sharing  their  attitudes  about 
the  method  of  peer  collaboration.  By  a  show  of  hands  vote,they  overwhelmingly 
endorsed  the  method  (37  to  3).  Group  consensus  focused  on  three  changes  to 
improve  the  classroom  application  of  the  study.  Students  fell  the  teaching  assistant 
should  assign  the  teams,  rather  than  allowing  the  students  to  choose.  They  also 
thought  more  time  should  be  allowed  to  consult  on  the  answers.  Students 
recommended  that  time  be  spent  in  class  for  training  and  instruction  in  group  work 
prior  to  working  in  teams. 


DISCUSSION 

The  significant  differences  in  achievement  by  the  peer  collaboration  group  that 
were  obtained  in  this  study,  combined  with  the  positive  attitude  findings,  are 
encouraging  for  additional  work  with  peer  collaboration.  Several  factors  may  have 
contributed  to  the  positive  results  for  peer  collaboration.  These  include  providing 
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and  explaining  the  answer,  obtaining  feedback  and  discussion,  and  playing  an 
active  role  in  learning. 

Students  working  with  partners  interacted  by  giving  and  explaining  their  answer, 
as  well  as  by  receiving  their  partner's  answer.  This  kind  of  interaction  may  also 
provide  :  experience  necessary  for  developing  the  kinds  of  consultative  skills 
needed  in  the  accounting  profession.  The  peer  collaboration  practice  mode  may 
have  provided  the  possibility  for  further  processing  of  information.  Kulhavy,  et  al 
(1988)  found  that  greater  time  spent  encoding  information  into  working  memory 
results  in  deeper  processing  and,  in  turn,  better  recall  of  information.  The  peer 
collaboration  mode  certainly  offered  the  opportunity  for  more  active  processing  of 
information,  which  could  have  led  to  better  recall  by  the  peer  collaboration  group. 

Another  important  aspect  of  the  study  is  the  possible  power  of  collaboration  after 
working  alone.  Students  initially  worked  alone  and  then  consulted  with  their 
partner.  This  form  of  peer  collaboration  or  consulting  combines  individual 
learning  with  group  interaction,  which  may  have  more  application  to  the  business 
world  which  the  students  plan  to  enter.  Patten  and  Doyle  (1990)  urge  accounting 
educators  to  fashion  the  accounting  education  experience  to  include  more  situations 
where  students  learn  to  work  together. 

Learning  to  collaborate  may  also  provide  the  learner  with  a  greater  sense  of  active 
participation  in  the  subject  matter.  The  aspect  of  being  engaged  in  the  class 
activities  is  increased  when  student  interaction  can  be  fostered  (Webb  and  Cullian, 
1983).  In  many  accounting  courses,  the  pace  of  content  coverage  is  deemed  to  be 
too  fast  by  the  students,  but  they  are  hesitant  to  say  so  in  class.  However,  once  the 
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students  are  more  actively  involved  in  peer  collaboration,  they  are  also  more 
willing  to  ask  questions  of  the  instructor. 

A  good  by-product  of  active  learning  may  be  a  more  positive  attitude  towards 
learning,  on  the  part  of  both  instructor  and  student.  Positive  attitudes  from  this 
study  appeared  to  encourage  more  student  involvement  and  participation  during 
class  discussions.  Students  felt  they  learned  more  by  working  together  in  class, 
which  might  also  provide  more  relevance  and  confidence  to  students.  Peer 
collaboration  may  provide  a  source  of  continuing  motivation  for  the  students. 

The  teaching  assistants  also  felt  the  use  of  peer  groups  increased  their  enjoyment 
of  teaching.  Both  teaching  assistants  continued  to  use  the  peer  collaboration 
practice  mode  throughout  the  rest  of  the  semester,  and  received  positive  student 
feedback  on  it  on  the  student  evaluations  at  the  end  of  the  semester. 

Based  on  the  results  from  this  study  and  due  to  the  major  curriculum  changes 
being  made  in  the  School  of  Accountancy,  peer  collaboration  is  now  being  included 
as  a  standard  class  practice  method,  with  much  success.  The  new  introductory 
courses  will  have  the  majority  of  all  class  activities  conducted  as  collaborative 
work.  Additionally,  the  use  of  peer  collaboration  is  being  investigated  for  the  new 
computer  iab  course. 

The  content  organizers  on  the  practice  worksheets  did  not  result  in  a  statistically 
significant  difference,  although  the  difference  did  approach  significance.  Whether 
such  an  approach  is  effective  is  still  unclear.  Perhaps  if  students  had  been  trained 
in  how  to  use  the  organizers,  a  stronger  effect  may  have  occurred.  Ellis,  et  al 
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(1986)  distinguish  between  specific  and  generic  organizers  and  suggest  that  more 
structured  instruction  is  needed  for  the  learners  to  effectively  use  content 
organizers. 

The  content  in  a  number  of  subjects  involving  applied  skills  often  lends  itself  well 
to  a  peer  collaboration  method.  For  example,  mathematics,  statistics,  lab  sciences, 
even  spelling,  could  use  this  method  for  class  practice.  The  results  of  this  study 
suggest  peer  collaboration  is  effective  in  producing  a  higher  level  of  learning  and 
appealing  to  the  students.  Students  showed  higher  levels  of  achievement  under 
peer  collaboration  and  Ihey  had  significantly  more  positive  attitudes  toward  this 
practice  mode. 

Classroom  application  and  research  study  continue  to  increase  in  the  area  of 
cooperative  learning.  The  present  study  raises  questions  that  are  appropriate  for 
further  investigation  in  peer  collaboration.  Does  peer  collaboration  work  better 
when  more  time  is  allowed  for  the  consulting  phase  of  practice?  Is  peer 
collaboration  or  consulting  a  way  to  combine  individual  accountability  and  team 
work,  thus  simulating  the  working  situations  many  accounting  students  will 
encounter  upon  graduation?  Would  training  in  the  most  effective  ways  to  work 
with  a  partner  increase  the  learning  benefits? 

The  results  from  this  study  indicate  that  peer  collaboration  is  an  effective  method 
for  use  in  the  technical  areas  of  accounting.  Further  research  and  applied  study 
using  peer  collaboration  can  help  us  to  understand  its  most  effective  uses  to 
improve  accounting  instruction. 
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table  i 

Test  Mean  Scores  and  Standard  Deviation  by  Treatment 


CONTENT  PRACTICE  MODE 

ORGANIZERS  Peer  Individual  Totals 

Organizers 

M  28.5  24.4  27.8 

S  7.6  6.6 

No  Organizers 

M  26.0  22.4  25.2 

5  _L2  10.2   

Mean  Totals  27.3  23.2  26  5 
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BACKGROUND 

The  onset  of  the  information  age  has  made  it  imperative  that  learners  develop  higher-order  thinking  skills.  Research 
is  just  beginning  to  reveal  die  full  power  of  multimedia  technologies  to  address  this  need  as  it  focuses  on  the 
amalgamation  of  multimedia  attributes  rather  than  on  the  significance  of  any  one  characteristic.  Multimedia 
technologies  can  contribute  to  the  development  of  higher-order  thinking  skills  because  they  facilitate:  learning  via 
structured  discovery,  student  motivation,  multiple  learning  styles,  the  navigation  of  web-like  representations  of 
knowledge,  learner  authoring  of  materials,  the  collection  of  rich  evaluative  information,  and  collaborative  inquiry. 
The  educational  reform  movement  is  providing  momentum  for  change  at  the  same  time  that  multimedia  devices  are 
finally  becoming  affordable  to  schools.  It  is  within  the  context  of  this  evolving  educational  environmmt  that  we 
undertake  the  Multimedia  and  Thinking  Skills  Project  in  order  to  demonstrate  a  new  paradigm  for  inquiry-based 
learning. 

InXune  of  1990,  the  research  group  at  George  Mason  University*  s  Center  for  Interactive  Educational  Technology 
(CBET)  began  designing  a  multimedia  prototype  to  foster  higher-order  thinking  skills  in  the  social  studies.  As  an 
initial  step  toward  that  objective,  Hie  Civil  War  Interactive  Project  using  Km  Burns'  documentary,  The  Civil  War, 
as  the  core  of  a  multimedia  database  produced  a  design  demonstration.  This  computer-based  instructional  system 
teaches  a  systematic  process  with  which  to  develop  higher-order  thinking  while  learning  how  to  engage  in  hi  storicsl 
inquiry.  With  funding  from  die  Corporation  for  Public  Broadcasting  and  technical  assistance  from  Apple  Computer, 
the  team  took  the  first  steps  toward  the  ultimate  goal  of  creating  a  generic  instructional  shell.  This  shell  will  be 
carefully  conceived  so  that  it  empowers  learners  by  giving  them  a  tool  that  promotes  the  development  of  thinking 
skills  as  they  acquire  bodies  of  knowledge.  We  believe  that  this  model  of  inquiry  can  be  easily  adapted  to 
multimedia  content  across  a  range  of  disciplines,  from  science  and  mathematics  to  the  social  sciences  and 
humanities. 

As  this  initial  project  has  progressed,  four  themes  are  providing  a  long-range  focus  for  our  work: 

*  The  multimedia  shell  for  teaching  thinking  skills  is  adaptable  to  material  from  a  variety  of  disciplines. 

*  Design  heuristic  are  emerging  about  how  to  design  multimedia  tools  that  give  teachers  leverage  on 
the  difficult  issues  of  cognition,  learning  styles,  motivation,  cooperative  learning,  evaluation,  and  the 
unique  needs  of  at-risk  students. 

*  Hie  quality  of  multimedia  databases,  particularly  the  quality  of  the  video,  is  sn  important  factor  in 
creating  effective  multimedia  prototypes  that  provoke  higher-order  thinking. 

*  Professional  development  activities  that  utilize  innovative  distance  learning  technologies  such  as 
on-line  databases,  teleconferences,  and  computer  networks  are  central  to  successful  utilization  of 
multimedia  products. 
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RATIONALE 


In  a  world  where  the  amount  of  daia  increases  exponentially  each  year,  a  major  challenge  for  schools  is  to  prepare 
learners  to  access  and  use  information  effectively.  Information  management  skills  have  never  been  more  vital  for 
students  entering  the  work  force.  The  computer  has  become  an  integral  part  of  doing  science  and  is  quickly  becoming 
a  fact  of  everyday  life.  Computers  generate  vast  amounts  of  technical  data  and  the  emerging  information  networks 
provide  access  to  overwhelnang  amounts  of  information.  However,  learners  frequently  become  lost  in  a  morass  of 
data  and  information  and  without  higher-order  thinking  skills,  they  cannot  synthesize  large  volumes  of  information 
into  overarching  knowledge  structures.  Unfortunately,  rich  technologies  such  as  multimedia  are  often  being  used  only 
to  provide  access  to  even  more  data  in  which  learners  can  drown,  thus  exacerbating  current  problems  rather  than 
contributing  to  solutions. 

We  bel if  vf  the  same  technologies  that  are  swamping  learners  in  data  can  help  them  master  die  flunking  skills  that  will 
promote  the  synthesis  of  information.  This  requires  a  refocusing  of  current  uses  of  multimedia  in  die  curriculum,  from 
engines  for  transmitting  massive  amounts  of  data  to  tools  for  structured  inquiry  based  on  higher-order  flunking .  Such 
sn  approach  should  be  equally  applicable  to  science,  mathematics,  the  social  sciences,  and  humanities.  The  best  role 
for  multimedia  in  schools  is  not  to  augment  data  delivery  in  conventional  instruction,  but  instead  to  foster  a  new  model 
of  teaching/learning  based  on  learners9  navigation  and  creation  of  knowledge  webs  through  a  formal  inquiry  process. 
Dede  (1990b)  suggests  that  higher-order  thinking  skills  for  structured  inquiry  are  best  acquired  where: 

*  Learners  construct  knowledge  rather  than  passively  ingest  information; 

*  Learning  is  situated  in  real-world  contexts  rather  than  based  in  artificial  environment  like 
eod-ofchapter  textbook  questions; 

*  Sophisticated  information-gathering  tools  are  used  to  stimulate  learners  to  focus  on  testing  hypotheses 
rather  than  on  plotting  data; 

*  Multiple  representations  for  knowledge  are  used  to  help  tailor  content  to  suite  individual  learning 
styles; 

41       There  is  collaborative  interaction  with  peers,  similar  to  team-based  approaches  underlying  today's 
science; 

*  Individualized  instruction  targets  teacher  intervention  to  assist  each  learner  in  solving  current 
difficulties;  and 

41       Evaluation  systems  measure  complex,  higher-order  skills  rather  than  simple  recall  of  facts. 


DESIGN  ELEMENTS 

This  project  has  identified  eight  elements  to  be  addressed  in  the  design  of  effective  instructional  multimedia  products: 
explicit  instruction,  modeling,  tutoring/coaching,  student  control,  collaborative  learning,  equity,  the  quality  of  the 
database,  and  professional  development.  The  generic  computer  shell  will  have  five  generic  features:  the  IBI  (Inquiry 
Bureau  of  Investigation),  Guided  Tours,  Dr.  Know,  the  Production  Console,  and  Custom  Tours.  The  IBI  is  literally 
an  iconic  bureau;  by  opening  each  of  its  drawers,  students  receive  explicit  instruction  on  the  steps  of  inquiry.  Guided 
Toun  model  the  inquiry  process  by  taking  student  through  the  database  on  carefully  design  pa&s  that  pose  questions 
and  compel  them  to  evaluate  the  extent  to  which  the  data  they  encounter  helps  to  answer  these  questions.  Dr.  Know 
is  the  context-sensitive  coach  or  tutor  who  helps  students  develop  data-gathering  and  inetacognitive  skills.  Custom 
Tours  is  a  non-structured  access  system  to  the  data  that  allows  free-form  searches  baaed  on  students*  interests  and 
are  in  the  students*  control .  Ths  Production  Console  gives  students  the  tools  with  which  to  manipulate  the  information 
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in  the  database  to  make  report*  and  create  their  own  tours.  It  it  not  only  a  device  for  student  control  but  it  also 
facilitates  collaborative  learning.  Other  design  element  such  as  the  quality  of  the  database,  equity,  and  professional 
development  are  discussed  as  environmental  factors  that  must  be  considered  if  successful  utilization  of  the  shell  is  to 
be  accomplished. 

Explicit  Instruction  -  Inquiry  Bureau  of  Investigation  (EM) 

The  IBI  is  literally  an  iconic  bureau;  by  opening  each  of  its  drawers,  students  receive  instruction  in  the  steps  of 
inquiry.  Because  higher-order  thinking  is  neither  instinctive  nor  developed  as  a  result  of  teaching  only  content, 
providing  opportunities  in  subject  matter  instruction  for  students  to  use  thinking  skills  is  not  enough  (Feuerstein,  1980; 
Perkins,  1987;  Sternberg,  1984;).  Instead,  students  need  continuing,  deliberate,  and  explicit  instruction  in  how  to  use 
inquiry  skills  —  not  in  a  decontextualized  manner,  but  simultaneously  with  their  striving  to  master  subject  matter 
(Salomon,  1991,  Whimbey  and  Whimbey,  1975).  Additional  studies  of  database  use  in  social  studies  (Ehman,  Glenn, 
Johnson,  and  White,  1992;  White,  1987)  and  hypertext  use  in  science  education  (White,  1989)  support  the  view  that 
effective  use  of  information  by  learners  requires  an  array  of  thinking  skill  supports.  In  a  study  of  word  processing 
as  a  way  to  improve  writing,  Zellermayer,  Salomon,  Oloberson  &  Givon  (1991)  indicate  that  courseware  with  guided 
instruction  and  continuous  high-level  cognitive  help  contributed  to  substantial  improvement  sustained  over  tune  and 
across  technologies. 

Modeling  -Guided Tours 

Guided  Tours  model  historic  and  scientific  inquiry.  They  are  purposeful  environments  that  deliberately  present 
contradictory  observations,  reports,  or  accounts  that  inspire  students  to  invent  additionri  questions,  hypothesize 
answers,  and  identify  potential  sources  of  data.  We  have  incorporated  into  our  design  an  anthropomorphic  coach,  Dr. 
Know,  who  acts  in  Guided  Tours  to  facilitate  learners  developing  inquiry  skills. 

As  Scardamalia,  Bereiter,  McLean,  Swallow,  Woodruff,  (1989)  describe,  students  require  extensive  scaffolding  to 
sustain  active,  constructive  approaches  to  learning.  Their  approach  centers  on  procedural  facilitation  of  learning  — 
not  machine-based  intelligence,  but  implicit  structure  and  cognitive  tools  that  enable  learners  to  maximize  their  own 
intelligence  and  knowledge.  In  the  context  of  multimedia,  procedural  facilitation  requires  the  instructional  system  to 
present  exemplary  knowledge-structuring  architectures,  illustrative  representational  formats,  and  prompts  that 
encourage  expert  information  managing  strategies.  In  our  design,  the  Guided  Tours  provide  the  first  two  of  these 
strategies  for  enhancing  learning,  and  Dr.  Know  provides  the  third. 

Tutoring/  Coaching  —  Dr.  Know 

In  our  prototypes,  resident  scholar  or  context-sensitive  coach,  Dr.  Know  can  be  called  upon  to  assist  students  in 
developing  data  gathering  and  metcognitive  skills  at  each  stop  along  the  Tour.  Laurel,  Oren  &  Don  (1990)  describe 
a  variety  of  ways  that  interface  agents  serving  as  guides  can  reduce  the  cognitive  load  for  users  of  multimedia 
systems.  In  Guided  Tours,  Dr.  Know  gives  context-sensitive  advice  on  how  to  explore  a  cluster  of  data,  how  to  think 
through  issues  in  an  inquiry  situation,  and  what  questions  may  be  most  productive  to  pursue.  In  the  IBI,  Dr.  Know 
gives  systematic  instruction  on  the  steps  of  inquiry  within  the  context  of  the  tour.  As  Beyer  (1990)  notes,  instruction 
clout  thinking  skills  at  the  time  that  they  are  contextually  needed  to  achieve  subject  matter  mastery  motivates  students 
to  acquire  these  skills  and  enhances  the  quality  of  subsequent  learning.  White  (1987, 1989)  also  notes  the  importance 
of  the  interactive  coach  in  supporting  skill  development. 

In  many  ways,  we  are  wrestling  with  the  same  issues  as  the  Cognition  and  Technology  Group  at  Vanderbilt  (1990) 
who  are  examining  the  concepts  of  anchored  instruction  and  situated  cognition.  They  are  creating  instructional  systems 
for  various  domains  that  permit  students  and  teachers  to  experience  inquiry  through  the  actions  of  an  expert.  Our  Dr. 
Know  is  an  expert  scientist  or  historian  who  takes  learners  (whom  the  Vanderbilt  Group  might  describe  as  apprentices) 
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along  on  an  investigation.  Through  modeling,  Dr.  Know  helps  learners  develop  their  alalia  and  knowledge.  This 
investigative  letting  for  anchoring  instruction  in  an  authentic  context  enhances  learner  motivation  and  makes  the  skills 
more  likely  to  generalize  into  real-world  environments. 

For  subject-independent  material  such  as  the  IBI,  Dr.  Know*!  help  transfers  across  disciplinary  domains.  For  specific 
Tours  in  a  particular  subject,  this  context-sensitive  help  must  be  customized  from  generic  templates  by  the 
instructional  design  team  creating  the  multimedia  materials  for  that  domain.  Dr.  Know  does  not  have  generative 
capabilities,  so  is  not  an  intelligent  coach;  the  complexity  of  developing  a  diaciplin^iiidependeiit,  glaes-box, 
knowledge-based  system  would  exceed  the  benefits  such  a  feature  could  provide.  Because  the  inquiry  model  is 
geoeralizable  across  disciplines,  our  preliminary  work  indicates  that  small  canonical  variations  of  prompts  make  Dr. 
Know  easily  adaptable  to  a  wide  range  of  subject  domains.  Unlike  the  group  at  Vanderbilt,  we  have  also  incorporated 
explicit  as  well  implicit  instruction  in  inquiry. 

Student  Control  -  Custom  Tours 

The  environment  we  are  developing  supports  instruction  in  which  students  have  some  freedom  of  choice  —  including 
producing  their  own  reflective  multimedia  materials  as  an  outcome  measure  indicating  mastery  —  and  in  which  they 
work  in  small  groups  using  collaborative  inquiry  to  explore  and  master  multimedia  material.  In  Custom  Tours  students 
can  use  the  facilities  of  the  Production  Console  (described  below)  to  create  their  own  path  through  the  data. 
Once  students  fed  they  know  the  inquiry  process,  they  are  encouraged  to  design  their  own  Custom  Tours.  As  students 
take  more  responsibility  for  their  own  learning,  Dr.  Know  prompts  them  to  state  Thoughtful  Questions  and 
hypotheses  that  will  help  them  establish  what  Scaidamalia  et  al.  (1989)  term  goal  orientation.  A  major  objective  of 
the  goal  orientation  strategies  Dr.  Know  conveys  is  to  de-emphasize  the  tendency  of  naive  learners  (and  teachers)  to 
rehearse  or  memorize  information;  instead,  stress  is  placed  on  skills  that  allow  data  to  be  reconfigured  as  needed. 

Helgeson  (1987)  establishes  the  importance  of  student  freedom,  choice,  and  the  use  of  small  groups  as  effective  in 
improving  scientific  problem-solving  behaviors.  However,  while  freedom  for  students  to  navigate  and  explore 
knowledge  webs  is  important,  our  prototypes  will  also  provide  extensive  implicit  structure  via  Guided  Tours  through 
the  multimedia  database.  As  Rivers  and  Vockell  (1987)  indicate  in  their  study  of  computerized  simulations  and 
problem  solving  skills,  students  using  guided  versions  of  simulations  surpassed  those  using  unguided  versions  on  tests 
measuring  cognitive  processes  and  critical  thinking. 

Student  Control  -  The  Production  Console 

Beyond  the  cluster  of  capabilities  associated  with  Dr.  Know  and  the  IBI,  the  multimedia/thinking  skills  shell  enables 
learners  to  produce  their  own  multimedia  presentation  and  Custom  Tours.  The  Production  Console  is  a  tool  to  help 
students  construct  both  Custom  Tours  built  around  their  own  Thoughtful  Questions  and  Reports  that  document  their 
proof/disproof  of  the  hypothesis  from  a  Guided  Tour.  In  the  Production  Console,  students  take  data  in  various  forms 
—  video,  audio,  graphic,  animation,  text  —  that  they  have  gathered  in  their  Journal  and  manipulate  them.  Learners 
can  add  additional  data  external  to  the  prototype,  can  analyze  their  journal  entries  based  on  the  inquiry  process  in  the 
IBI,  and  can  author  their  own  tours  to  augment  those  provided  with  the  instructional  system. 

As  Donoho  (1986)  notes,  participation  in  audiovisual  production  contributes  to  student  learning  in  curriculum  areas 
such  as  language  arts,  social  studies,  and  science.  Students  engaged  in  these  production  activities  demonstrated 
improvements  in  self-concept,  motivation,  creativity,  and  attitude.  In  addition,  these  students  increased  their 
involvement  with  print  media. 

The  Production  Console  is  important  for  another  reason:  students  use  this  feature  to  demonstrate  their  knowledge, 
their  ability  to  analyze  date  and  state  conclusions,  and  their  skills  in  applying  what  they  have  learned.  To  date,  little 
work  has  been  done  on  evaluating  student  outcomes  from  instruction  with  multimedia  but  a  US  Office  of  Technology 
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Assessment  report  (1987)  indicates  that,  if  technologies  are  to  succeed  in  schools,  teachers  must  have  better  methods 
of  evaluating  student  outcomes.  An  objective  of  our  project  is  to  develop  improved  evaluative  measures  that  can  help 
teachers  assess  learners'  mastery  of  inquiry  skills. 

Cooperative  Learning 

Capabilities  such  ss  the  Production  Console  support  cooperative  learning  because  students  can  easily  share  and  display 
data,  comment  on  drafts,  and  keep  collective  records  of  their  inquiry  process.  The  production  of  multimedia  tours 
is  a  particularly  powerful  vehicle  for  collaboration  because  learners  must  pool  unique  individual  strengths  (editing 
video,  scanning  images,  digitalizing  sound,  creating  animation,  writing  text)  to  develop  a  successful  product.  While 
there  has  been  limited  research  on  collaborative  or  cooperative  learning  in  multimedia  environments,  Adams,  Carlson, 
Hamm's  (1990)  review  of  the  research  indicates  that  collaborative  learning  with  interactive  technologies  leads  to 
higher  scores  on  measures  of  content  knowledge  and  observation  skills,  as  well  as  high  degrees  of  learner  motivation 
and  satisfaction. 

Equity 

Equity  issues  are  an  important  theme  underlying  our  approach  to  moving  beyond  traditional  evaluation  methods  into 
innovative  strategies  that  assess  higher-order  thinking  skills  and  complex  behaviors  denoting  mastery.  Research 
suggests  that  current  uses  of  information  technology  in  education  may  be  widening  the  gaps  between  rich  and  less 
affluent  schools  and  between  high-achieving  and  at-risk  students  (Ascher,  1984;McPhail  1985;  Morgan,  1987).  Public 
schools  in  impoverished  districts  and  small  rural  schools  are  more  likely  to  have  high  student/computer  ratios,  and 
wealthy  schools  tend  to  offer  computer-based  enrichment  activities  while  financially  stressed  schools  focus  on 
computer-assisted  remedi*tion-  Educational  policy  goals  are  clear  in  mandating  that  instructional  technologies  be  made 
equally  available  to  all  students  and  in  supporting  efforts  to  address  the  needs  of  historically  underrepresented 
population*. 

In  a  recent  summary  of  research  on  computers  and  at-risk  students,  Rockman  (1991)  notes  that  well-designed 
educational  technology  both  enhances  the  achievement  of  at-risk  learners  and  improves  their  attitudes  toward  school 
and  the  subject  matter  they  study.  Even  though  many  of  these  studies  warn  of  novelty  effects  on  achievement  and 
attitude  outcomes,  the  research  seems  to  indicate  that  at-risk  students,  using  information  technology,  experience  gains 
in  linguistic  fluency,  subject  matter  comprehension,  problem  solving  abilities,  and  mathematics  skills  Atutudinal 
measures  demonstrate  higher  levels  of  self-confidence  and  self-esteem,  enthusiasm  for  the  use  of  technology,  and 
appreciation  of  the  ability  to  control  the  pace  of  their  learning  (Clariana  and  Smith,  1988;  David,  1986;  Dieesner 
*  WtUcer  l9SS>  E**"™  J«»W.  1985;  Ely,  1984;  Henderson,  Landesman,  &  Kwhuch,  1983;  Jacobs 
1985;  Laboratory  of  Comparative  Human  Cognition,  1989;  Mehan,  Moll,  &  Riel  1985;  Miura,  1987;  Payne,  1986- 
Rosegrant,  1985;  and  Rosa,  Smith,  Morrison,  Ericson,  &  Kitabchi,  1989). 

However,  tittle  research  has  been  done  that  documents  how  multimedia  technology  can  be  designed  to  effectively  build 
on  the  strengths  and  motivations  of  at-risk  students.  We  believe  that  our  multimedia/thinking  skills  approach  offers 
promise  for  motivating  students  who  find  conventional  instructional  materials  uninteresting  because  they  are 
predominantly  textual/symbolic,  passive,  linear,  centered  on  memorizing  data,  and  remote  from  issues  learners  find 
meaningful  m  their  own  Uvea.  In  particular,  we  hypothesize  that  multiple  representations  of  knowledge  to  address 
different  learning  styles,  collaborative  inquiry  to  build  on  differential  individual  strengths,  and  active  production  of 
knowledge  webs  could  reach  students  now  uninvolved  in  classroom  instruction. 

Quality  of  the  Database 

As  Kozma  (1991)  notes,  the  instructional  effectiveness  of  any  one  medium  varies  with  the  nature  of  instruction  and 
toe  nature  of  the  learner.  Wc  are  concerned  with  constructing  a  multimedia  database  that  stimulates  blinking  and 
supports  a  variety  of  diverse  learners  as  they  construct  their  own  cognitive  webs.  We  draw  from  a  very  high  quality 
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video  database  for  The  Civil  War  interactive  Project,  Ken  Burns*  film  The  Civil  War,  and  augment  it  with  a  rich 
collection  of  primary  source  document*  including  photographs,  diaries,  audio  tapes  and  maps.  During  the  development 
of  the  project  we  will  attend  to  the  extent  to  which  the  nature  and  quality  of  the  database  affects  learning  for  a  wide 
array  of  learners. 

Particular  attention  will  be  paid  to  the  role  of  video.  Research  into  the  instructional  effectiveness  of  video  alone  has 
not  produced  clear  results,  as  Johstone  (1987)  notes,  film  and  video  are  more  works  of  art  than  feats  of  engineering 
thus  making  it  difficult  to  quantify  the  qualities  of  film  and  video  that  make  it  affective  in  instruction.  However,  we 
do  know  that  well-produced  video  helps  to  motivate  students;  Gibbon  &  Hooper  (1986)  found  that  video  was  a 
powerful  force  in  motivating  students  to  want  to  learn  more  science  and  mathematics. 

In  some  previous  educational  multimedia  products  there  has  been  a  tendency  to  utilize  "repurpoeed  "  video  segments 
(segments  originally  intended  for  a  completely  different  use)  or  to  use  segments  of  lower  quality.  Semper  (1990) 
argued  that  the  quality  of  the  video  segments  employed  in  multimedia  products  affects  the  level  of  learning  attained 
by  users  and  that  higher  quality  segments  encourage  greater  involvement  and  thus  moie  learning.  Naimaik  (1990), 
Chen  (1990),  and  Ambron  (1990)  agreed,  with  each  citing  the  relationship  between  the  quality  of  the  video  and  the 
quality  of  the  learning  experience.  Mountford  (1990)  accounted  for  this  relationship  by  suggesting  that  "drama  has 
the  power  to  engage  audience  members  both  emotionally  and  intellectually"  (p. 21),  while  Florin  observed  that 
"because  the  footage  is  emotionally  compelling,  you  want  to  leara  more  on  the  subject.  ...  But  what  is  most 
significant  here  is  that  the  story  gives  you  a  reason  to  study  the  materials"  (1990,  pp.  36-37).  In  light  of  these 
concerns,  our  project  draws  upon  powerful  end  informative  footage  with  high  production  values,  seeking  to  use 
powerful  video  segments  as  vehicles  for  learner  involvement.  Hie  demonstration  prototype  of  the  Civil  War 
Interactive  Project  utilized  the  powefull  footage  from  Kan  Burns*  series,  Th*  Civil  War,  to  draw  students  into  the 
adventure  of  historical  inquiry.  Similarly,  the  other  prototypes  in  the  Multimedia  and  Thinking  Skills  Project  will 
utilize  well-produced  footage  that  will  draw  learners  into  the  study  of  science,  mathematics  or  social  sciences.  We 
view  video  producers  and  others  with  extensive  video  experience  as  key  members  of  our  development  team. 

Professional  Development 

Excellent  curricula  that  rely  on  non-traditional  models  of  teaching/learning  tend  to  fail  unless  support  is  provided  for 
teachers  to  master  the  new  paradigm  for  instruction  (Havelock,  1970,  1973).  As  the  prototypes  are  developed,  we 
will  assess  the  types  of  scaffolding  necessary  for  teachers  to  use  these  materials  successfully  in  a  typical  classroom 
without  special  resources  or  expertise.  The  professional  development  methods  we  envision  include  both  conventional 
strategies  (workshops,  self-instructional  kits)  and  innovative,  telecommunications-based  approaches  such  as 
videoconferences  and  networked  virtual  communities  for  providing  support  across  barriers  of  distance. 

The  central  focus  underlying  all  these  professional  development  efforts  is  to  help  teachers  reconceptualize  the  purpose 
of  disciplinary  instruction:  from  memorization  of  facts  to  mastery  of  thinking  skills  and  knowledge.  Even  with  a 
sophisticated  multimedia  prototype  to  scaffold  their  instruction,  teachers  need  extensive  support  to  restructure  their 
pedagogical  approaches  toward  inquiry-oriented  learning.  In  a  study  of  teachers*  critical  thinking  and  performance 
in  social  studies,  Fontana  (1980)  noted  that  success  in  improving  students*  critical  thinking  skills  may  depend  on 
improving  teachers*  skills  in  integrating  higher  cognitive  activities  into  their  instructional  planning  and  implementation. 

One  product  of  this  project  will  be  a  plan  that  both  details  the  scope  and  type  of  professional  development  needed  to 
enable  teachers  to  use  our  multimedia/ thinking  skills  materials  and  assesses  opportunities  to  use  emerging  information 
infrastructures  to  support  ongoing  instructional  innovation.  Dwyer,  Ringstaff,  &  Sandholtz  (1991)  note  that  effectively 
integrating  technology  into  the  classroom  requires  changes  in  teachers*  instructional  behaviors.  Over  time,  these 
behavioral  changes  occur  as  teachers  reflect  on  their  own  beliefs  about  learning  and  instruction  and  as  the 
administrative  structure  of  the  school  shifts  to  accommodate  these  changes.  These  researchers  also  note  that  the  nature 
of  the  support  needed  for  teachers  as  they  engage  in  the  process  of  change  alters  at  different  stages  of  implementation. 
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For  example,  in  early  stage*  of  innovation  teachers*  needs  center  on  their  concerns  about  the  technology  itself;  skill 
development  is  the  most  important  type  of  support.  As  adoption  and  adaptation  proceed,  teachers  increasingly  need 
opportunities  to  think  about  instructional  issues  and  to  engage  in  ongoing  dialogues  about  their  experiences. 

Hunter  (1990)  points  to  the  potential  of  computer-mediated  communications  to  support  teachers  as  they  experience 
fundamental  change  in  their  instructional  strategies  and  beliefs.  Fontana  and  Ochoa  (1985)  caution  that  administrators 
need  to  provide  supportive  environments  for  change  and  require  their  own  training  and  support  system.  Bruder  (1991) 
outlines  some  of  the  current  local  and  regional  activities  that  use  telecommunications  to  train  teachers.  Dede  (1990a) 
describes  trends  that  are  making  distance  learning  via  telecommunications  a  reality  in  professional  development.  All 
this  research  underscores  the  need  for  the  development  of  creative  technology-based  strategies  for  professional 
development. 

The  Thinking  Skills  Project  views  students  as  tourists  through  multimedia  databases.  Just  as  tourists  make  choices 
about  how  they  will  explore  different  sights,  users  of  our  multimedia/thinking  skills  shell  can  decide  how  to  explore 
their  cognitive  environment.  They  may  choose  one  of  several  Guided  Tours  or  they  can  explore  the  database  via 
Custom  Tours.  A  reflective  context  for  learning  is  created  in  the  exemplary  Guided  Tours;  these  provide  stimulating 
environments  within  which  students  become  actively  engaged  in  learning  the  subject  domain  while  receiving 
context-sensitive  instruction  on  the  inquiry  process.  As  students  proceed  through  Guided  or  Custom  Tours,  they  are 
able  to  call  up  their  on-line  tutor,  Dr.  Know,  to  help  them  evaluate  information  in  various  form.  During  their  tours 
they  collect  information  in  their  electronic  Journal  which  they  take  to  the  Production  Console.  It  is  here  in  the 
Production  Console  where  they  consult  the  1BI  and  make  certain  they  have  followed  all  the  steps;  they  are  then  ready 
to  create  reports  or  design  their  own  tours. 

Because  it  is  so  difficult  to  envision  how  the  feature  we  have  just  discussed  will  work  together  to  promote  the 
development  of  higher-order  thinking,  we  have  included  a  short  scenario  that  describes  one  way  in  which  The  Civil 
War  Interactive  Multimedia  Product  might  be  used  in  the  classroom.  This  is  just  one  of  many  optimistic  visions  of 
the  potential  classroom  application  of  our  work. 

THE  APPLICATION  OF  THE  MULTIMEDIA  SHELL:  THE  CIVIL  WAR  INTERACTIVE 

Ron,  Juan,  and  Kim  are  three  students  from  Ms.  Regis's  class  who  have  chosen  to  develop  a  cooperative  inquiry 
project  on  the  Civil  War.  Ms.  Regis  has  suggested  some  potential  topics  the  students  might  select  and  has  encouraged 
them  to  pursue  other  topics  once  they  can  frame  these  in  the  form  of  Thoughtful  Questions.  The  students  were  present 
when  Ms.  Regis  briefed  the  class  on  using  The  Civil  War  Interactive;  while  Juan  and  Kim  are  excited  and  interested, 
Ron  is  not  sure  he  wants  to  do  much  work.  He  really  doesn't  see  how  the  Civil  War  relates  to  him  and  is  not  so  sure 
about  this  new  technology.  While  Juan  and  Kim  are  interested  in  the  topic  they  are  not  entirely  comfortable  with  the 
inquiry  process  or  how  to  operate  the  instructional  system. 

Juan  takes  the  lead  with  the  mouse  and  enters  the  system  hesitantly.  On  the  opening  screen,  he  identifies  himself  as 
a  group  leader  for  the  "Yankees*  and  starts  by  reviewing  the  Guided  Tour  that  Ms.  Regis  used  in  her  class  discussion. 
They  choose  the  Tour  which  is  guided  by  the  Thoughtful  Question  "How  are  peoples  lives  affected  by  war?".  They 
open  the  Guide  book  and  see  the  four  stops:  Prewar  Life  in  the  South,  Prewar  Life  in  the  North,  The  Portrait 
Gallery,  and  Life  After  the  War.  They  decide  to  go  further.  Ron  grumbles,  "this  is  boring."  He  would  rather  do 
something  else;  after  all,  what  does  the  Civil  War  have  to  do  with  him?  He  only  half  listens  as  Juan  and  Kim  discuss 
the  kinds  of  answers  they  might  expect  to  find  in  response  to  the  Thoughtful  question. 

He  finally  speaks  up  when  Juan  and  Kim  are  trying  to  decide  whether  to  go  on  this  tour.  Ron  thinks  that  they  should. 
At  the  next  screen  the  group  sees  two  hypotheses.  The  instructions  on  the  screen  indicate  that  they  must  choose  one. 
Ron  is  not  even  sure  what  an  hypothesis  is,  or  why  he  should  care.  But  as  he  looks  at  the  second  one,  his  interest 
is  peaked  and  he  encourages  the  students  to  choose  the  hypothesis  that  states: 
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Individuals  in  the  North  and  South  were  affected  differently  by  the  Civil  War  depending  on  their  economic, 
political,  social,  and  geographic  circumstances. 


They  then  decide  to  go  to  the  fin*  stop,  Prewar  Life  in  the  South.  They  are  unsure  which  of  the  aighta  to  visit.  Mrs. 
Regit  comet  over  to  tee  if  they  are  having  trouble  and  they  tell  her  they  are  not  certain  what  to  look  at  firtt.  She 
tuggetta  that  they  might  want  to  check  with  Dr.  Know.  By  clicking  on  the  icon  at  the  bottom  of  the  screen, ;  second 
screen  pops  up  and  lists  three  kinds  of  advice  you  can  get:  "E*  or  exploration  advice,  "T"  or  thinking  process  advice, 
or  "Q"  for  more  questions  to  consider.  Ron  wants  to  check  the  exploration  advice.  Juan  says  OK  and  turns  the  mouse 
over  to  him.  He  clicks  and  a  screen  then  pops  up  that  explains  the  relationship  between  the  information  found  at  the 
different  eights  and  suggests  that  they  begin  with  the  Agriculture  and  Plantation  System. 

Kim  is  interested  in  the  kind  of  thinking  advice  they  will  receive  and  asks  Ron  to  go  back  to  Dr.  Know  to  get  that 
advice.  Here  they  get  some  suggestions  about  how  they  should  be  thinking  about  this  information  in  relationship  to 
their  hypothesis.  Back  at  the  first  screen  of  the  first  stop  they  take  Dr.  Know ' s  advice  and  go  to  the  plantation  system. 
Another  screen  comes  up  with  classifications  of  primary  sources  they  can  examine  on  Slaves,  Plantations,  Social  Life, 
and  Political  Life.  They  choose  to  look  at  more  information  on  slavery  and  find  that  they  can  explore  documents, 
video  shows,  or  photographs.  Kim  wants  to  see  the  comprehensive  video  essay.  She  remembers  that  Mrs.  Regis 
showed  some  of  it  in  class  and  she  wants  to  see  it  again.  The  video  monitor  pulls  up  the  footage  from  The  Civil  Wan 
the  music  is  Jacob's  Ladder  and  then  they  hear  the  crickets  as  the  camera  explores  the  photographs  of  slaves  working 
in  the  field.  Then  the  narrator  begins  to  speak  as  the  camera  focuses  intently  on  the  photographs  of  individual  slaves 
on  the  auction  block  and  slaves  in  chains. 

Ron  is  now  rivetted.  He  was  not  in  class  the  day  Ms.  Regis  gave  the  introduction.  He  had  no  idea  how  horrible 
slavery  was.  He  had  never  been  this  close  to  it  before.  Kim  comments  that  slavery  is  no  longer  just  a  word  in  a 
book.  It  was  a  horrible  way  to  live.  Ron  is  listening  to  the  narrator  give  the  statistic  on  the  number  of  African 
Americans  who  lived  as  slaves  and  what  life  was  like  in  a  slave  cabin.  He  can't  take  it  all  in.  He  stops  the  video  and 
goes  back  and  calls  up  the  section  on  slave  cabins  again  and  hears  and  sees  one  more  time  how  awful  the  ordeal  really 
was. 

They  are  about  to  go  into  more  depth  on  the  issue  of  slavery  by  examining  the  pension  record  of  an  African  American 
woman,  Susan  Drane,  when  a  prompt  reminds  them  that  they  are  on  a  Guided  Tour  and  they  might  want  to  review 
where  they  have  come  so  far  and  perhaps  note  other  questions  they  have  on  this  topic  in  their  Journal  for  latter 
consideration. 

The  students  agree,  but  they  have  forgotten  the  hypothesis.  They  think  a  minute  and  Kim  realizes  that  all  they  need 
to  do  is  to  click  on  hypothesis  at  the  top  of  the  screen.  Up  it  pops.  She  now  wants  her  turn  on  the  mouse.  She 
clicks,  up  pops  the  Thoughtful  Questions.  Now  they  are  not  at  all  sure  what  to  do.  Juan  suggests  that  they  ask  Dr. 
Know  who,  in  turn,  suggests  that  they  review  the  pension  record  carefully  looking  for  information  on  how  the  war 
affected  African  Americans  in  the  South,  and  that  they  place  that  in  their  Journal.  He  notes  that  if  they  go  to  the 
Journal  it  will  give  them  directions  about  how  to  capture  video  footage,  copy  text  from  the  document,  and  how  to 
write  their  own  notes.  They  take  the  advice  and  work  on  composing  their  joint  observations. 

They  return  to  the  pension  record  and  because  it  is  13  pages  long,  they  decide  to  print  it  out  and  Kim  agrees  to  read 
at  home  and  mark  the  sections  that  they  should  keep  in  their  Journal.  She  is  really  interested  in  how  African 
American  women  lived  under  slavery  and  during  Reconstruction.  She  thinks  about  the  fact  that  her  ancestors  must 
have  lived  like  this. 

The  students  continue  their  Guided  Tour  over  the  next  two  days.  They  explore  the  video  segments  on  African 
American  troops,  they  meet  John  Boston  and  make  a  copy  of  his  letter.  They  read  part  of  Mary  Chestnut's  diary  and 
then  hear  how  she  used  the  diary  to  help  her  cope  with  th*  war.  They  hear  die  words  of  U.S.  Grant,  W.T.  Sherman, 
and  Robert  E.  Lee  and  are  struck  by  the  fact  that  their  letters  home  sound  much  like  any  other  soldier  who  wants  a 
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war  to  end  and  to  come  home  to  those  who  love  them.  They  alto  hear  and  read  the  words  of  Abraham  Lincoln  and 
come  to  know  the  pain  of  a  mm  who  it  trying  to  hold  his  country  together;  yet  not  compromise  an  important  principle 
like  freedom*  They  learn  how  many  of  the  founding  fathers  like  Thomas  Jefferson  were  just  as  tormented  by  slavery 
as  Lincoln,  while  others  felt  that  die  issue  of  states  rights  was  more  important. 

They  have  kept  good  records  in  their  Journal  and  now  have  much  to  sort  out.  They  move  the  Journal  to  the 
Production  Console.  Now  what?  They  click  on  the  Dr.  Know  and  he  suggests  that  they  review  the  slept  of  inquiry 
by  clicking  on  the  drawers  of  the  IBI  (Inquiry  Bureau  of  Investigation).  They  skip  the  Question  and  Hypothesis  drawer 
and  go  directly  to  the  drawer  marked  Testing  Data. 

They  review  the  data  they  have  gathered  and  then  double-check  their  Journal  taking  note  of  the  patterns.  They  then 
begin  to  apply  the  questions  in  the  Testing  Date  drawer  and  discuss  their  immediate  project. 

Is  your  data  relevant  or  necessary  to  proving  or  disproving  the  hypothesis? 
Do  you  have  sufficient  data? 

What  is  the  source  of  each  piece  of  data?  Is  the  source  credible?  Is  it  reliable? 
Does  any  piece  of  data  incorporate  bias  or  narrow  points  of  view? 
Does  each  piece  of  data  make  a  persuasive  and  logical  argument? 
Are  stereotypes  represented? 

The  students  are  not  familiar  with  the  full  meaning  of  each  of  the  elements  of  the  Dsta  Test.  However,  by  clicking 
on  any  step,  they  receive  a  brief  description  of  that  stage  of  the  inquiry  process  and  illustrative  examples  of  how  to 
apply  that  type  of  data  test.  The  students  complete  the  second  day  of  their  cooperative  inquiry  by  eliminating  some 
of  the  evidence  they  collected  and  by  beginning  to  arrange  the  remainder  in  s  form  that  addresses  their  hypothesis. 

Ron  and  Kim  have  found  some  other  charts  and  data  in  books  in  the  library;  they  scan  these  into  their  Journal.  The 
students  now  feel  that  they  have  enough  data  to  begin  writing  their  conclusions.  They  refer  occasionally  to  the 
Concluding  drawer,  which  helps  them  state  their  conclusions  based  on  the  data  they  have  gathered.  The  group 
eventually  comes  to  agreement  with  regard  to  whether  they  have  proven  their  hypothesis  true  or  false.  They  begin 
to  compose  their  multimedia  report. 

They  realize  in  the  middle  of  the  process  that  history  and  multimedia  are  more  fun  than  they  expected,  especially  since 
each  member  of  the  group  can  build  on  his/her  strengths:  Ron  is  the  technical  expert,  Juan  the  visual  editor,  Kim  the 
writer.  In  the  process  of  using  the  Production  Console  to  prepare  the  presentation,  they  realize  how  much  they  have 
learned.  They  discuss  the  fact  that  they  had  never  thought  about  what  life  was  really  like  under  slavery-  Ron  is 
particularly  amazed  because  he  never  liked  history.  Kim  wants  to  find  out  more  about  African  American  Women  and 
Juan  enjoys  sorting  through  the  information  and  carefully  selecting  what  will  become  part  of  their  presentations.  They 
all  fed  more  confident  about  starting  their  next  research  project  in  history.  They  all  feel  that  they  have  a  deeper  sense 
of  what  the  Civil  War  was  about. 

They  are  anxious  to  show  their  work  to  Ms,  Regis,  who  is  very  impressed  with  what  they  have  done.  She  reviews 
the  students'  inquiry  process  and  conclusions  and  then  has  them  go  through  the  inquiry  check-list  that  is  part  of  the 
evaluation  strategy  for  student  products.  She  then  is  ready  to  review  their  presentation  and  listen  to  their  analysis. 

(This  scenario  was  adapted  from  The  Civil  War  Interactive  Demonstration  and  Project  Design  Document) 
CONCLUSION 

The  Multimedia  and  Thinking  Skills  project  described  here  is  an  ambitious  one.  It  combines  the  development  of  an 
instructional  multimedia  products  with  the  search  for  new  knowledge  about  the  potential  of  the  technology.  While 
looking  to  past  research  to  inform  design  decisions,  the  project  is  not  bound  by  existing  models.  The  prototype 
products  that  we  hope  to  produce  will  be  more  than  rich  multimedia  databases  with  sophisticated  navigational  systems 
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but  will  provide  a  glimpse  into  the  future  and  help  us  all  understand  more  clearly  how  to  hameaa  the  instructional 
power  of  the  emerging  multimedia  technologies. 
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Jn  the  Spring  of  1990,  a  team  of  researchers  at  Northern  Illinois  University  began  to 
compile  findings  from  a  study  investigating  the  most  critical  audience  factors  impacting  the 
design  and  delivery  of  instruction  for  international  learners  in  higher  education  settings  in  the 
United  States. 

Results  of  early  findings  provided  a  direction  for  additional  inquiry  that  is  still 
continuing  on  the  problems  and  challenges  that  face  both  learners  from  other  cultures  and  their 
U.S.  instructors  in  university  environments. 

Findings  are  being  incorporated  in  small  bulletins  called  "Culturefax"  that  are  being 
distributed  to  higher  education  institutions  and  other  interested  parties.  These  bulletins  note 
special  cultural  factors  that  may  prompt  U.S.  instructors  to  become  more  sensitive  and  more 
effective  in  multicultural  classrooms.  These  bulletins  highlight,  for  example,  if  a  certain 
nationality  is  generally  more  comfortable  in  group  or  team  learning  activities  or  in  more 
individualized  programs. 

Other  findings  that  have  been  useful  in  developing  the  highlights  have  been:  the 
presentation  style  and  even  dress  of  the  instructor,  mediated  materials  preferences,  personal 
treatment  of  the  learners,  use  of  glossaries  to  supplement  existing  texts  for  learners  with 
language  handicaps,  timing  and  pace  of  instruction,  and  a  dozen  other  factors. 

It  is  important  to  discuss  the  findings  of  the  studies  and  to  review  the  methodologies. 
The  researchers  want  to  know  the  strengths  and  weaknesses  of  the  techniques  and  usefulness  of 
the  results.  The  study  and  findings  have  been  examined  in  some  detail  and  comments  on  the 
procedure*  invited  from  the  participants  in  several  professional  presentations. 

The  connection  between  the  findings  and  the  development  of  the  bulletins  have  been 
incorporated  in  revised  bulletins  and  recommendations  from  all  ihe  audiences  will  be 
considered  for  possible  revision  to  continuing  research  and  publication  of  the  "Oilturefax". 

"Culturefax"  is  continuously  reviewed  with  the  intent  of  incorporating  the  audience 
critique  in  subsequent  publications.  Those  who  are  critiquing  the  bulletins  have  a  multicultural 
educational  background  and  act  as  consultants  to  this  project.  It  is  hoped  that  this  will  be  a 
multicultural  and  interactive  activity  that  will  provide  a  arena  for  the  researchers  and  the 
participants  to  actively  continue  the  research  process  and  thus  play  a  major  role  in  the  revision 
and  reshaping  of  the  final  publications.  This  process  suggests  a  robust  means  of  validating  the 
research,  the  bulletins  and  subsequent  multicultural  instructional  techniques. 

Sample  handouts  that  capture  the  research  and  examples  of  the  multicultural 
instructional  bulletins  are  provided,  and  cover  such  themes  as: 

1 .  Platform  skills  in  the  multicultural  education  setting 

2.  Selection  of  appropriate  media 

3.  Pacing  and  timing  of  instruction 

4.  Treatment  of  learners 

5.  Special  cultural  factors  or  religious,  political  or  social  considerations 

6.  Instruction  in  English  factors 

7.  Subject  matter  issues 

8.  Classroofti  environments. 

Subsequent  reviews  will  focus  on  summarizing  revisions  and  recommendations  for 
adjusting  the  on-going  research  and  on  considering  suggestions  for  improving  the  format  of 
Culturefax". 
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ORIGINATION  OF  THE  THEORY 
OF  CROSSCULTURAL 
INSTRUCTION 


Key  theoretical  constructs  that  generated  the  research  were  suggested  by  the  work  of 
Philip  R.  Harris  and  Robert  T.  Moran  and  reported  in  a  book  entitled:  Managing  Cultural 
Differences:  High-Performance  Strategies  for  Today's  Global  Manager,  2nd  Edition,  1987, 
Gulf  Publishing  Company.  ( ISBN  0-87201-161-5  ). 

In  the  text,  the  authors  posited  several  categories  of  cultural  conditions  that  can  impact 
learning  by  adult  internationals  in  U.S.  classrooms.  These  became  the  general  areas  of 
inquiry  that  the  study  attempted  to  address. 

The  conditions  are  as  follows: 

SENSE  OF  SEL*  AND  SPACE 

COMMUNICATION  AND  LANGUAGE 

DRESS  AND  APPEARANCE 

FOG2  AND  EATING 

TIME 

RELATIONSHIPS 
VALUES  AND  NORMS 
BELIEFS  AND  ATTITUDES 
MENTAL  PROCESS  AND  LEARNING 
WORK  HABITS 

Additionally,  Harris  and  Moran  stated  that  the  cultural  impact  on  learning  could  be 
further  refined  to  cover  educational  sub-heads: 

TEACHER  PERCEPTION 

ENGLISH  PROFICIENCY 

CLASSROOM  ATTITUDES  OR  BEHAVIOR 

PERSONAL  VALUES 

OTHER 

These  categories  and  subheads  were  analyzed  and  formed  into  statements  that  could  by 
tested  for  specific  cultural  groups.  The  initial  intent  was  to  clarify  these  statements  to 
determine  if  they  would  generate  responses  that  could  be  developed  into  hypotheses.  The  final 
outcomes  would  not  be  facts,  findings  or  generalizationa  but  rather  testable  statements  derived 
from  the  collection  of  answers. 

The  categories  and  subheads  were  presented  to  representative  international  students  at 
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Northern  Illinois  University  and  the  response*  tabulated  and  condensed  to  form  the  hypotheses. 

This  approach  was  also  recommended  in  a  dissertation  by  Dr.  Jeanne  Hites  entitled: 
Design  and  Delivery  of  Instruction  for  International  Trainees:  a  Case  Study,  Northern 
Illinois  University,  Spring,  1990.  Dr.  Hites  suggested  the  development  of  testable  statements 
for  A.T.  &  T.  instructors  who  were  involved  in  the  teaching  of  technical  switching  systems. 
She  also  employed  a  process  termed:  "triangulation"  to  confirm  that  the  statements  from 
international  respondents  were  as  valid  as  possible. 

So,  using  "triangulation"  that  included  student  comments,  previous  instructors' 
comments  and  any  written  or  anecdotal  dat*  available,  she  generated  hypotheses  that  could  be 
tested  by  in-service  instructors  at  A.T.&T.  This  puts  research  on  international 
teaching/learning  at  the  appropriate  and  most  relevant  level:  the  classroom;  and,  provides  the 
investigator  with  very  reliable  and  valuable  hypotheses  that  can  be  tested  and  possibly 
subsequently  lead  to  improved  classroom  instruction  and  learning.  It  also  attempts  to  avoid  the 
so-called,  "Margaret  Mead"  error  of  generalizing  too  extensively  from  respondents  without 
other  types  of  corroborating  evidence. 

This  "triangulation"  method  using  students  as  respondents,  instructors  as  informants 
and  written  or  anecdotal  information  in  support  of  each  statement  or  hypothesis  was  utilized  in 
this  study.  As  mentioned  earlier,  the  statements  have  been  put  into  bulletins  entitled 
"Culture fax",  and  will  be  provided  as  requested  to  interested  instructors,  instructional 
designers  and  others. 

Currently  three  cultural/national  audiences  have  been  examined  and  pilot  "Culturefax" 
bulletins  produced.  These  are:  Philippines,  Thailand  and  Indonesia.  In  addition  to  the  printed 
bulletins  a  "LINKWAY",  that  is  IBM  interactive  instructional  program,  is  available  for  the 
Philippines  version  of  "Culturefax".  Future  interactive  programs  are  being  developed  for 
several  other  "Culturfctax"  as  well.  In  summary  for  publication  this  Spring  are:  Malaysia, 
People's  Republic  of  China,  and  Kenya.  Later  bulletins  will  cover:  Taiwan,  South  Africa  and 
Egypt. 

The  initial  audiences  were  selected  as  a  function  of  being  the  predominant  nationalities 
served  by  the  faculty  of  Instructional  Technology  at  Northern  Illinois  University,  DeKalb, 
Illinois. 

Plans  included  distribution  to  other  units  in  the  university  and  to  the  public  at  large  for 
a  small  fee  to  cover  printing,  handling  and  mailing.  Additionally,  research  on  other  specific 
audiences  will  be  undertaken  on  request  and  workshops  on  preparing  U.S.  instructors  for  the 
international  classroom  provided  by  contract. 

The  following  sample  of  the  Philippine  (Filipino)  "Culturefax-  provides  a  good 
example  of  the  results  of  this  international  learner  study. 

RECOMMENDATIONS  FOR  THE  FILIPINO  CLASSROOM 

1.  Expect  to  be  treated  with  "regard  and  respect". 

2.  Don't  rely  on  instruction  based  on  argumentation. 

3.  Be  polite  and  somewhat  formal. 

4.  Provide  learning  activities  that  focus  on  reading  and  writing  but  expect  fairly  fluent 
spoken  English  with  some  accents. 

5.  Avoid  intense  or  "heavy"  discussions. 
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6.  Keep  changes,  confrontations  and  disturbances  to  a  minimum. 

7.  Give  recognition,  incentives  and  encouragement  whenever  possible. 

8.  Relate  to  the  student  in  a  "warm  and  personal"  way. 

9.  Use  lecture  and  rather  direct  instruction. 

10.  Test  with  essay  and  objective  techniques. 

USING  "CULTUREFAX" 

Recommendations  for  the  classroom  are  drawn  from  comments  and  observations  by  . 
respective  international  students  and  their  U.S.  instructor  as  well  as  reports     from  other 
documentation.  This  does  not  guarantee  that  they  are  appropriate  in  all  cases.  The  data  that 
support  the  recommendations  are  valuable  but  subject  to  errors  that  accompany  any  research 
that  relies  heavily  on  informants  and  interpretations. 

Use  the  recommendations  as  experiments.  Consider  the  recommendations  as  classroom 
hypotheses  that  you  can  confirm  or  refute  for  your  specific  teaching  environment.  You  are  the 
besti evaluator  of  these  suggestions  for  your  learners.  If  you  find  that  the  points  are  not  useful 
to  you,  disregard  them  and  write  to  us  and  tell  us  your  findings.  It  is  important  to  provide  the 
mo*  effective  and  current  recommendations  possible.  Your  observations  will  play  a  part  in 
updating  and  revising  future  "culturefax".  Thank  you  for  your  help. 

OTHER  FACTORS  THAT  DEFINE  THE  FILIPINO  LEARNER 
Teacher  Perception 

Filipino  students  respect  authority.  Instructors  arc  considered  authority  figures  in  their 
respective  fields.  Students  look  up  to  teachers  with  high  regard  and  respect.  Students 
are  expected  not  to  question,  disagree,  argue,  or  challenge  their  teacher's  viewpoints. 

Students  address  their  teachers  in  a  polite  and  formal  way.  They  use  titles  such  as: 
-Sir  Mam,  Mr.,  Mrs.,Dr,  etc."  Being  in  an  informal  classroom  setting  would  make  a 
student  uncomfortable.  Students  also  need  to  feel  that  they  arc  .respected  by  their 
teachers. 

English  Language  Proficiency 

Most  Filipino  students  speak  English  well  because  they  have  had  their  basic  background 
courses  taught  in  English.  But  English  is  not  the  native  language. 

Some  students  may  have  difficulty  expressing  ideas  in  English.  Filipino  students  read 
and  write  English  better  than  they  speak  it. 

Some  students  have  distinct  accents  when  speaking  English. 

Classroom  Attitude 

Students  do  not  easily  volunteer  ideas,  express  opinions,  or  raise  questions. 

Filipino  students  are  not  very  verbal,  forward,  or  assertive  and  frank.  They  have  a 
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tendency  to  be  shy  or  quiet.  They  prefer  writing  things  down.  They  avoid  heavy 
discussions. 

There  is  also  the  tendency  to  be  submissive  by  agreeing,  avoiding  confrontations, 
changes  and  disturbances.  Students  want  to  have  a  smooth  interpersonal  relationship 
with  others. 

Personal  Values 

Students  like  to  be  given  compliments,  incentives,  encouragement,  recognition,  and 
acceptance. 

They  like  those  teachers  who  show  interest  in  their  progress  and  relate  to  them  in  a 
warm  and  personal  way. 

Be  careful  of  the  comments  and  jokes  made  in  class.  They  might  be  more  insulting 
than  funny. 

Others 

Students  prefer  essay  type  of  questions/examinations  or  a  combination  of  objective  and 
essay. 

They  are  used  to  lecture  type  of  interaction.  Filipinos  can  adapt  to  their  environment 
easily. 

Sense  of  Self  and  Space 

Filipinos  are  very  private  and  do  not  open  up  easily  to  other  people. 
They  like  to  work  on  their  own. 
They  do  not  volunteer  information. 

They  function  well  in  situations  when  they  are  given  their  own  space,  independence. 
Filipinos  are  mostly  sensitive  and  emotional  human  beings. 
Communication  and  Language 

Talalog  is  the  national  language,  but  English  (which  is  the  medium  of  instruction)  is 
frequently  used  especially  in  the  business,  education  and  the  mass  media. 

There  also  exists  urban  and  rural  slangs  and  jargons;  also  regional  dialects. 

For  Filipinos,  their  written  English  is  better  than  their  spoken  English. 

Dress  and  Appearance 

Differences  exist  between  urban  and  rural  attire.  Urban  styles  are  Western  influenced, 
while  rural  people  wear  simple  work  clothes. 

Most  business  settings  require  certain  dress  codes. 

Filipinos  are  conscious  of  their  physical  appearance  and  dressing  up  well  relates  to 
one's  status  and  good  family  breeding. 
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Food  and  Eating  Habits 

Regional  differences  are  evident  even  in  food. 

Rice  is  the  staple  food;  food  is  generally  not  that  spicy;  people  are  fond  of  sweets  for 
dessert. 

People  generally  use  forks  and  spoons  but  some  still  prefer  using  their  hands. 

"Burping"  is  not  a  sign  of  being  disrespectful  but  that  one  had  a  very  fine  meal. 

Some  don't  eat  meat  due  to  religious  beliefs. 
Time  and  Time  Consciousness 

The  Philippines  is  known  for  "Filipino  time"  -  not  being  punctual. 

Most  people  try  to  be  more  punctual  when  it  comes  to  business  meetings. 

"Man ana"  is  commonly  practiced v 
Relationships 

Strong  family  ties  are  evident  among  Filipinos. 

A  nuclear  family  may  even  include  the  grandparents,  cousins,  uncles,  etc.  and  other 
immediate  relatives  and  also  what  they  call  "adopted  relatives". 

Men/women  or  boy /girl  relationships  are  also  geared  towards  the  family. 

Fixed  marriages  are  not  prevalent  but  in  some  regions  parents  still  engage  in  this  set* 
up. 

Pre-  and  extra-marital  activities  and  living-in  are  taboo. 
Long  engagements  are  still  preferred. 

Manifestations  of  affection  in  public  is  considered  inappropriate. 
Values  and  Norms 

They  respect  elders  or  any  authority  figure. 

People  are  expected  to  behave  in  particular  ways  in  relation  to  their  economic  status, 
profession  or  educational  accomplishment. 

Beliefs  and  Attitudes 

Filipinos  are  predominantly  Catholics  in  religion. 

Religious  authorities  or  superiors  play  a  big  role  in  influencing  decision  making. 

There  also  exist  beliefs  in  the  supernatural,  in  superstitions,  fairies,  dwarves,  gods  and 
goddesses. 
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Menial  Process  and  Learning 

Early  in  life,  Filipino  children  are  taught  by  their  parents  basic  readiness  skills  needed 
in  school.  Depending  on  the  subject  matter,  teachers  engage  in  either  drill  and  practice 
or  rote  memorization.  Individualized  learning  dominates  the  classroom  setting 
encouraging  the  students  to  work  on  particular  tasks  at  their  own  pace/level.  As 
students  go  to  higher  levels  of  education,  teachers  encourage  student*  to  engage  more 
independent  and  creative  thinking. 

Work  Habits  and  Practices 

They  respect  their  superiors  and  tend  to  obey  and  follow  them  even  if  it  mwos  doing 
something  they  "don't  readily  believe  in  or  agree  with. " 

Some  don't  mind  working  extra  hours  or  working  late  or  even  working  during  the 
weekends  just  so  they  could  accomplish  a  particular  task. 

Most  prefer  the  onset  of  new  technology  in  the  workplace  while  others  still  prefer 
manually  operated  or  less  sophisticated  equipment.  Changes  are  not  easy  to  deal  with 
in  routine  activities. 

Nepotism  is  evident  and  payment  is  expected  for  favors  especially  in  conducting 
business  deals;  loyalty  to  job  is  evident  especially  in  the  educational  system. 

The  work  place  serves  as  a  social  arena. 

Organizations  often  plan  social  events. 


FINALLY 

It  is  very  important  to  validate  the  "Culturefax"  by  the  means  at  our  disposal.  It  is 
important  for  the  statements  or  hypotheses  to  be  valuable,  and  for  the  recommendations  to  be 
useful.  With  this  in  mind,  this  study  and  prospective  publications  are  being  presented  to 
scholars  of  international  education  at  several  professional  meetings  to  provide  additional 
insight  into  the  process  for  generating  the  statements  and  for  assessing  the  merit  of  the 
product.  The  observations  will  then  become  an  additional  and  very  important  aspect  of  the 
research  "triangulation". 

Your  comments  and  ideas  are  most  welcome  and  will  be  considered  in  the  revision  of 
the  process  and  the  resultant  "Culturefax". 


PLEASE  SEND  YOUR  COMMENTS 

CULTUREFAX 

David  G.  Gueulette 
L.E.RS. 
Northern  Illinois  University 
DeKalblL  60115 

815-753-0464 
FAX  815-753-2100 
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Abstract 

This  study  is  an  investigation  of  how  modeling  aids  learners  in  developing 
problem  solving  skills  within  a  computer  learning  environment  The  task  given 
to  45  subjects  was  to  solve  apime  in  the  computer  game,  Where  In  Time  Is 
Carmen  San  Diego.  Three  subject  groups  received  one  of  the  following 
treatments:  Expert  Model,  Novice  Model,  or  No  Model  Modeling  entailed 
subjects  viewing  a  computer-based  animated  and  narrated  movie  showing  how 
either  an  expert  or  novice  completed  the  task.  Results  of  the  study  indicate  that 
modeling  does  not  have  an  effect  on  the  performance  of  participants,  however  it 
did  contribute  to  the  development  of  cognitive  skills. 
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The  Effects  of  Modeling  to  Aid  Problem  Solving 
in  Computer-Based  Learning  Environments 

The  capabilities  offered  by  computer-based  learning  (CBL)  systems  have  led 
to  significant  innovations  in  instructional  delivery  systems,  specifically  in  the 
design  of  simulations,  demonstrations  and  drill-and-practice.  The  design  of 
many  CBL  programs  is  linear  and  systematic.  However,  new  paradigms  for 
implementing  CBL,  such  as  hypermedia,  require  new  design  models  and 
prescriptions  to  guide  the  development  of  instructional  systems. 

This  study  focuses  on  the  instructional  method  of  modeling  and  its 
application  in  a  CBL  environment.  It  is  imperative  that  the  importance  of 
modeling  is  not  misunderstood.  We  believe  that  there  is  a  need  for  a  more 
effective  way  to  facilitate  cognitive  learning  in  a  CBL  environment,  and  that 
modeling  delivered  by  computer  is  a  strategy  with  considerable  potential. 
Specifically,  we  want  to  determine  how  computer-based  modeling  of  strategies 
within  a  CBL  environment  influences  the  acquisition  of  cognitive  skills. 

Theoretical  Background 

The  history  of  modeling  includes  such  theorists  as  Miller  &  Dollard,  Bandura, 
and  even  Aristotle.  Their  work  focused  mainly  on  the  concepts  of  imitation  and 
observational  learning.  Aristotle  noted  that  an  individual,  "Is  the  most  imitative 
of  living  creatures,  and  through  imitation  learns  his  earliest  lessons;  and  no  less 
universal  is  the  pleasure  felt  in  things  imitated."  (Butcher,  1922,  p.  15)  More 
recently,  the  social  learning  theorists  Miller  and  Dollard  (1941)  offer  a  cogent 
description  of  the  role  of  imitation  and  social  learning,  emphasizing  that 
imitative  learning  is  an  acquired  social  behavior.  Bandura  (1971)  conducted 
extensive  research  on  psychological  modeling  in  the  behaviorist  realm. 
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A  comparison  of  definitions  of  modeling  is  useful  for  isolating  attributes  of 
this  instructional  treatment.  Bandura  (1971)  describes  modeling  using  a  broad 
range  of  terms,  "Among  the  diverse  terms  applied  to  matching  behavior  are 
,imitation,/  'modeling',  'observational  learning',  'identification',  'internalization', 
'introjection',  'incorporation',  'copying',  'social  facilitation',  'contagion',  'and  'role 
taking'"  (p.  3).  Romiszowski  (1981),  on  the  other  hand,  gives  an  apt  description 
of  modeling  as  the  showing,  not  telling,  a  learner  some  set  of  processes,  steps, 
skills  and  strategies  to  solve  a  task.  Davies  (1981)  and  Gagn6  (1987)  define 
modeling  simply  as  "demonstrations." 

More  recently,  Welsh  (1992)  differentiates  two  types  of  modeling,  behavioral 
modeling  and  cognitive  modeling.  We've  attributed  behavioral  modeling  to 
much  of  the  research  we've  illustrated  in  the  preceding  paragraphs.  Cognitive 
modeling,  on  the  other  hand,  incorporates  systematic  verbalization  on  the  part  of 
the  model  to  make  his  or  her  thinking  processes  visible  to  others.  Thinking  aloud 
and  mental  modeling  are  two  strategies  that  describe  what  occurs  during 
cognitive  modeling. 

Behavioral  Modeling  Research 

"Algorithms  guarantee  solutions:  if  properly  carried  out  they  always  yield  the 
correct  answer."  (Perkins,  1986:198).  Demonstrative  modeling  is,  in  essence,  an 
algorithm,  since  it  is  the  objective  of  the  modeler  to  show  the  learner  the  perfect 
solution  to  the  problem.  The  research  has  consistently  shown  that  demonstrative 
modeling  strategies  for  completing  tasks  have  yielded  greater  performance  than 
no  modeling  (Yaeshima,  Ninomiya,  Ohnogi  &  Oda,  1982;  LaNunziata,  Cooper  & 
Hill,  1985;  Rivera  &  Smith,  1987;  Baggett,  1987;  Smith  &  Hailey,  1988;  Robert  & 
Chaperon,  1989). 
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Smith  &  Hailey  (1988)  found  that  demonstrative  modeling  techniques 
resulted  in  better  performance  in  learning  breast  self-examination  over  narrative 
explanation  and  written  instructions.  In  their  experiment,  the  experimenters 
assigned  60  college  women  to  one  of  three  instructional  treatments:  actual  breast 
demonstration,  synthetic  breast  demonstration,  and  no  demonstration.  After  one 
month,  the  two  subject  groups  who  received  the  demonstration  treatments 
performed  better  than  the  control  group.  Yet  after  three  months,  all  differences 
between  experiment  and  control  subjects  disappeared. 

Baggett  (1987)  investigated  the  use  of  a  narrated  film  and  a  correctly 
assembled  object  as  stimulus  in  the  task  of  assembling  a  toy  helicopter  from  a 
number  of  given  pieces.  The  modeling  by  film  consisted  of  a  15  minute,  narrated 
8mm  film  showing  an  expert  naming  and  putting  together  the  pieces  to  assemble 
the  helicopter  correctly.  The  assembled  object  was  a  completely  assembled  toy 
helicopter.  Baggett  found  that  any  combination  of  film  and  the  assembled 
helicopter  resulted  in  greater  task  completion  than  either  treatment  in  isolation. 

Robert  and  Chaperon  (1988)  conducted  a  linking  experiment  between 
behavioral  and  cognitive  modeling.  In  their  experiment,  they  used  a  Piagetian 
horizontal  representation  water-level  task  with  college  women.  Experimenters 
randomly  assigned  women  to  one  of  three  groups:  cognitive  modeling, 
exemplary  modeling  and  no  modeling.  Cognitive  modeling  entailed  watching  a 
videotape  of  a  woman  as  she  correctly  solved  three  water-level  tasks.  During  her 
solution,  she  bom  demonstrated  and  verbalized  her  actions.  Exemplary 
modeling  used  the  same  videotape  of  the  woman  performing  the  task,  however 
she  never  voiced  any  part  of  her  solution.  The  no  modeling  group  received  no 
modeling  of  the  task.  Experimenters  immediately  evaluated  subjects  after 
treatment  on  their  ability  to  solve  a  similar  task  with  the  same  water  bottles  and  a 
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task  with  different  water  bottles.  Fifteen  days  later,  experimenters  conducted  a 
delayed  post-test  tc^measure  stability  of  the  learned  behavior. 

The  results  from  this  experiment  clearly  showed  the  impact  of  both  cognitive 
modeling  and  exemplary  modeling  in  enabling  women  to  master  the  water-level 
task.  However,  the  cognitive  modeling  component  appeared  to  be  the  critical 
component  that  fostered  improvement  in  the  learner. 

Cognitive  Modeling  Research 

There  is  still  the  question  we  raised  earlier  of  the  technique  one  uses  to  model 
cognitive  processes~i.e.,  think-aloud  or  mental  modeling.  Schoenfeld  (1985)  uses 
think-aloud  routines  to  model  the  cognitive  process  of  solving  math  problems. 
He  has  an  ongoing  contest  whereby  students  present  math  problems  for  him  to 
solve.  In  solving  the  problem,  he  thinks  out  loud,  articulating  his  strategies  for 
solving  the  problem.  In  essence,  he  is  showing  the  students  how  he  thinks  in 
seeking  a  solution  to  a  math  problem,  exposing  them  to  all  the  complexities, 
mistakes  and  pitfalls  one  must  deal  with  along  the  way. 

In  teaching  reading  skills  to  biology  students,  Mikulecky,  Clark  and  Adams 
(1989)  evaluated  modeling  as  a  means  of  coaching  college  freshmen  to  read  texts 
more  strategically.  In  particularly,  they  used  computer-based  presentations  of 
text  and  analysis  to  model  strategies  such  as  identifying  key  concepts,  writing 
linking  summary  statements  to  compare  and  contrast  key  concepts,  and 
graphically  mapping  relationships  among  key  concepts.  They  found  that 
modeling  these  reading  skills  for  biology  students  with  a  CBL  program  yielded 
higher  exam  scores  over  a  control  group  that  did  not  receive  the  CBL  iessons. 

Mandinach  (1987)  compared  discovery  learning  and  participant  modeling  as 
a  method  to  develop  strategic  planning  knowledge.  Two  groups  of  seventh  and 
eighth  grade  students  played  a  game  called  Wumpus.  The  experimenters  used  a 
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discovery  learning  paradigm  for  one  group,  while  the  other  group  received 
participant  modeling  that  systematically  modeled  appropriate  cognitive  skills 
and  strategies.  Students  in  the  participant  modeling  group  performed  better  in 
task  completion;  however  the  discovery  learning  group  performed  better  in  the 
transfer  of  skills  to  other  tasks. 

Synthesis 

Between  both  behavioral  modeling  and  cognitive  modeling,  the  key 
instructional  elements  that  comprise  models  are  demonstration  and 
verbalization.  While  demonstration  is  an  established  construct,  verbalization  is 
somewhat  more  undefined.  For  example,  in  Robert  and  Chaperon's  (1988) 
research,  verbalization  was  operationalized  by  describing  one's  actions  during  a 
perfectly  demonstrated  task.  On  the  other  hand,  Schoenfeld  (1985)  also 
verbalizes  his  solution  to  math  problems,  however  this  verbalization  includes  the 
mistakes  he  is  making  and  how  he  resolves  those  mistakes.  We  believe  the 
demonstration  of  mistakes  and  verbalization  of  how  to  recover  from  them  is  a 
more  effective  instructional  strategy  for  facilitating  the  development  of  cognitive 
skills  than  modeling  perfect  solutions.  Thus,  our  research  will  investigate  the 
differences  between  these  two  verbalization  strategies  in  the  context  of  cognitive 
modeling  within  CBL. 

Hypotheses 

On  the  basis  of  the  literature,  we  expect  our  study  to  replicate  the  findings  of 
prior  modeling  research. 

HI :  Participants  who  are  shown  a  model  of  how  to  perform  a  task  will 
perform  better  on  that  task  than  participants  who  are  not  shown  a 
model. 
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How  we  define  the  protocol  for  modeling  of  cognitive  processes  is  important 
to  developing  our  study.  Our  primary  concern  is  how  experts  think  about 
solving  tasks,  and  how  to  teach  those  thinking  skills  to  others. 

In  the  context  of  prior  modeling  research,  we've  developed  two  constructs  for 
modeling.  The  first  construct  is  the  expert  demonstration  model,  whereby  an 
expert  demonstrates  and  explains  how  to  perform  a  task  perfectly.  The  second 
construct  operationalizes  one  of  the  salient  features  of  Schoenfeld's  research, 
namely  the  modeling  of  how  to  deal  with  errors.  In  this  construct,  a  novice 
performs  a  task  while  an  expert  provides  commentary  on  the  novice's 
performance. 

The  showing  of  incorrect  solutions  is  an  important  distinction  to  make 
between  expert  and  novice  modeling.  Expert  moiling,  as  described  in  the 
literature,  tends  to  show  the  correct  performance  of  a  task,  whereas  novice 
modeling  shows  both  correct  and  incorrect  (with  recovery)  solutions.  People 
make  mistakes  every  day,  and  learning  how  to  recover  from  those  mistakes  is  an 
important  skill.  We  believe  the  observation  of  a  novice's  recovery  from  incorrect 
solutions  contributes  greatly  to  acquiring  cognitive  skills.  Therefore,  we  predict 
that  novice  modeling  will  result  in  successful  completion  of  a  task  over  the  expert 
model  since  the  participant  will  learn  how  to  avoid  and  recover  from  common 
mistakes  novices  tend  to  make. 

H2:  Participants  who  observe  a  novice  model  while  listening  to  an  expert 

comment  on  the  novice's  performance  will  perform  better  in  a  task 

than  participants  observing  an  expert  model. 
Performance  is  not  the  only  measure  of  success.  Our  interest  in  the  cognitive 
skills  a  learner  acquires  as  a  result  of  instruction  drives  our  belief  that  modeling 
will  have  a  positive  effect  on  the  key  strategies  a  participant  uses  in  solving  a 
task. 
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H3:  Participants  who  observe  a  model  will  use  key  task  strategies  more 
effectively  than  participants  who  do  not  observe  a  model. 

H4:  Participants  who  observe  the  novice  model  will  use  key  task 

strategies  more  effectively  than  participants  who  observe  the  expert 
model. 

Another  measure  of  a  learners  cognitive  skill  is  articulation.  Collins,  Brown 
&l  Newman  (1988)  suggest  that  having  a  learner  articulate  what  they  know,  by 
explaining  how  they  plan  to  solve  a  problem  or  why  they  applied  a  particular 
solution,  one  can  get  a  sense  of  the  cognitive  skills  they've  learned.  Thus,  we  will 
investigate  how  the  different  modeling  treatments  affect  the  participant's 
articulations. 

H5:  Participants  who  observe  a  model  will  have  stronger  articulations 
about  how  to  solve  a  task  than  participants  who  do  not  observe  a 
model. 

H6:  Participants  who  observe  the  novice  model  will  have  stronger 

articulations  about  how  to  solve  a  task  than  participants  who  observe 
the  expert  model. 

H7:  Participants  who  have  strong  articulations  will  use  key  task  strategies 
more  effectively  than  participants  who  have  weak  articulations. 

H8:  Participants  with  strong  articulations  about  how  to  solve  a  task  will 
perform  better  in  a  task  than  participants  with  weak  articulations. 

Method 

Design 

The  experiment  was  a  l-by-3  design.  There  was  one  variable,  modeling,  with 
three  levels.  The  levels  were  expert,  novice,  and  control  The  expert  level 
consisted  of  a  modeling  presentation  showing  an  expert  playing  Where  I|\  Time 
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Is  Carmen  Sarf  Diego  (WITICSD)  with  the  experts  vocal  narration.  The  novice 
level  consisted  of  a  modeling  presentation  showing  a  novice  playing  WITICSD 
with  an  expert's  vocal  analysis  of  the  novice's  performance.  The  control  level 
consisted  of  no  modeling  presentation. 

Participants 

Fifty-two  undergraduate  and  continuing  education  students  from  an 
introductory  computer  course  at  a  large  midwestern  university  signed  up  to 
participate  in  the  study.  The  course  instructor  awarded  extra  credit  to  the 
participants  for  participation.  We  screened  participants  for  prior  experience  with 
WITICSD.  None  of  the  participants  had  ever  played  WITICSD  or  any  of  the 
other  Carmen  San  Diego  games.  Seven  participants  failed  to  show  up  for  their 
appointment,  resulting  in  45  participants  who  contributed  data  to  the  study. 

Materials 

Materials  for  the  study  included  a  Macintosh®  SE  computer,  WITICSD 
software  and  documentation,  Microsoft  Word™  software,  custom-designed 
modeling  presentation  software,  written  instructions,  a  pad  of  paper,  and  a  pen. 

WITICSD  is  an  educational  software  game.  In  the  game,  the  player  takes  on 
the  role  of  a  detective  solving  a  crime.  Solving  the  crime  entails  interacting  with 
the  software  to  obtain  clues,  tracking  the  criminal  through  time  and  space, 
acquiring  an  arrest  warrant,  and  arresting  the  criminal.  Players  decipher  clues 
about  the  identity  of  the  criminal  and  the  destination  he  or  she  is  traveling  to  by 
using  the  New  American  Desktop  Encyclopedia,  which  comes  with  the  software. 
We  also  provided  the  standard  documentation  for  WITICSD,  which  includes  the 
encyclopedia,  instructions  on  how  to  use  the  software  and  hints  for  playing  the 
game. 
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In  order  for  participants  to  write  their  articulations,  we  used  Microsoft  Word 
software  for  word  processing.  Thus,  participants  wrote  their  articulations  using 
the  computer. 

The  custom-designed  modeling  presentation  software  consisted  of 
MediaTracks™-produced  "movies."  MediaTracks  enables  one  to  record  actions 
seen  on  the  computer  screen.  After  one  records  the  movie,  one  can  edit  and  add 
sound  to  the  movie.  The  result  is  an  animated,  narrated  presentation.  We 
produced  one  movie  for  presenting  the  study's  instructions  to  the  participant, 
one  movie  for  the  expert  treatment,  and  one  movie  for  the  novice  treatment. 

The  instruction  movie  was  3:17  (minutes:seconds)  in  length.  The  content  of 
the  movie  entailed  screens  of  text  that  illustrated  the  narration  of  instructions  for 
the  study  (see  Appendix  A).  We  also  provided  a  print  version  of  the  instructions 
to  the  participants. 

The  expert  movie  was  7:16  in  length.  The  content  of  the  movie  entailed  an 
expert  playing  WITICSD,  showing  the  perfect  solution  for  playing  the  game.  The 
expert's  narration  consisted  of  noting  the  strategies  and  heuristics  he  used  while 
playing  the  game  (see  Appendix  B). 

The  novice  movie  was  10:54  in  length.  On  the  basis  of  observations  of  novices 
playing  WITICSD,  we  identified  the  most  common  mistakes  novices  make.  The 
content  of  the  movie  was  a  novice  playing  WITICSD.  An  expert  narrated  as  the 
novice  performed.  The  narrator  noted  the  successful  and  unsuccessful  strategies 
the  novice  was  using  (see  Appendix  C). 

It  is  important  that  we  make  clear  the  distinction  between  the  expert  and 
novice  movies.  The  expert  movie  emphasized  a  perfect  path  solution  to  the  task 
and  illustrated  advanced  techniques  for  solving  the  problem  whereas  the  novice 
movie  illustrated  errors  and  their  outcomes  and  solutions  in  the  course  of 
playing  the  game. 
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Additional  materials  for  the  study  included  a  blank  pad  of  legal  paper  and  a 
pen. 

Procedure 

Pre-Treatment  Activities.  Participants  signed  up  for  individual,  one  hour 
time  slots  over  a  one  month  period  in  the  late  fall.  To  confirm  their 
appointments,  we  contacted  each  participant  by  phone  or  E-mail.  We  told 
participants  that  they  would  be  user-testing  some  educational  software. 

We  conducted  the  study  in  the  conference  room  of  a  university  building.  The 
computer  was  on  the  conference  room  table.  The  blank  legal  pad  and  the  pen 
were  to  the  left  of  the  computer,  and  the  instructions,  program  documentation 
and  encyclopedia  were  to  the  right  of  the  computer.  The  experimenter  sat  to  the 
back  and  right  of  the  participant  in  order  to  observe  all  of  the  participant's 
actions. 

When  the  participant  arrived  at  the  testing  site,  the  experimenter  explained 
the  nature  of  the  experiment  to  the  participant  as  well  as  issues  of  confidentiality 
and  voluntary  participation.  In  order  to  decrease  anxiety,  the  performance  of  the 
software,  not  the  participant,  was  emphasized.  The  experimenter  told 
participants  that  they  would  be  testing  some  educational  software  and  that  our 
interest  was  in  the  software's  performance,  not  theirs. 

Next,  the  experimenter  pointed  out  the  resources  the  participant  had  at  his  or 
her  disposal  throughout  the  study.  The  experimenter  showed  the  participant  the 
documentation,  instructions,  encyclopedia,  notepad  and  pen. 

The  experimenter  then  played  the  instruction  movie  for  the  participant.  Prior 
to  playing  the  movie,  the  experimenter  reminded  participants  that  they  had  a 
printed  copy  of  the  written  instructions  that  they  could  use  at  any  time  during 
the  experiment  or  to  follow  along  during  the  movie. 
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Since  the  focus  of  this  study  was  on  problem  solving  strategies  and  not 
procedural  knowledge  of  using  the  software  and  hardware,  we  wanted  to  make 
sure  all  participants  were  comfortable  in  the  procedural  aspects  of  the  Macintosh 
and  WITICSD  before  receiving  treatment.  After  the  participant  had  viewed  the 
instruction  movie,  the  experimenter  would  explain  the  basic  use  of  the  computer 
and  the  mouse  pointing  device.  This  required  that  the  participant  demonstrated 
proficiency  in  using  the  mouse  to  interact  with  the  computer.  We  always 
observed  accomplishment  of  this  exercise. 

In  order  to  make  the  participant  familiar  with  WITICSD  user  interface,  the 
experimenter  conducted  a  practice  task.  In  this  task,  the  experimenter  guided  the 
participant  in  using  the  WITICSD  software  by  exploring  one  location.  The 
experimenter  explained  how  to  click  the  various  buttons,  obtain  clues,  and  travel 
from  one  location  to  another  while  the  participant  clicked  the  appropriate 
buttons.  The  experimenter  told  the  participant  that  this  was  a  practice  task  for 
their  familiarity  only  and  he  encouraged  them  not  to  be  concerned  about  solving 
the  task  successfully.  Participants  could  ask  questions  about  the  functional  use  of 
the  software  and  hardware  only  during  this  practice  task. 

These  orienting  activities  and  associated  practice  not  only  helped  the 
participants  learn  about  how  to  use  the  software,  but  helped  form  the  context  for 
the  task  itself,  especially  the  modeling. 

Treatments.  The  experimenter  randomly  assigned  participants  to  one  of  the 
three  treatments,  expert,  novice  and  control.  For  the  expert  and  novice 
treatments,  the  experimenter  prefaced  the  showing  of  the  appropriate  movie 
with  the  statement  that  the  participant  will  now  see  a  short  movie  that  will  give 
the  participant  more  information  about  playing  WTTICSD.  The  experimenter 
encouraged  the  participant  to  do  nothing  else  but  watch  the  movie.  The  control 
treatment  group  did  not  watch  any  movie. 
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Articulation  Task.  Upon  completion  of  the  treatment,  the  experimenter 
opened  the  word  processing  software  and  instructed  the  participant  to  answer 
the  question,  "Explain  to  a  friend  how  to  play  Where  In  Time  Is  Carmen  San 
Diego.11  Each  participant  had  five  minutes  to  answer  the  question.  When  the 
participant  finished  his  or  her  answer,  the  experimenter  sav  sd  the  answer  in  a 
file  named  after  the  participant. 

Game  Task.  At  this  point,  the  participant  played  a  single  game  of  WITICSD. 
The  experimenter  set  up  the  software  so  that  all  participants  played  the  same 
game,  the  stolen  Sultan's  Howdah.  In  order  to  avoid  participants  who,  lacking 
motivation  to  work  efficiently,  simply  randomly  pressed  buttons  to  finish  the 
experiment  quickly,  we  offered  a  $10.00  prize  to  the  participant  who  completed 
the  game  with  the  most  remaining  chronotime  (a  measure  of  efficiency  used  in 
the  game). 

Dependent  Measures.  During  the  task,  the  experimenter  recorded  data  using 
a  data  tabulation  instrument  (See  Appendix  D).  The  experimenter  recorded  data 
for  each  location  the  participant  visited. 

•  Location /Date.  The  location  and  date  the  participant  navigated  to 
while  playing  WITICSD.  For  example,  France,  1892. 

•  WITICSD  Chronotime.  The  chronotime  displayed  on  the  game  screen 
was  recorded  when  the  participant  first  entered  a  new  location.  When 
the  participant  made  a  successful  arrest,  the  ending  chronotime  was 
also  recorded.  Unsuccessful  arrests  resulted  in  a  chronotime  of  0  being 
recorded. 

•  Actual  chronological  time.  Time  was  recorded  to  the  second  at  the 
beginning  and  conclusion  at  each  location  and  for  the  entire  game. 

•  Witness.  One  point  (or  hash  mark)  was  recorded  each  time  the 
participant  went  to  the  witness  for  clues.  If  the  participant  had  the 
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option  to  obtain  more  information  while  viewing  the  witness,  an 
additional  mark  of  "m"  indicated  that  the  participant  viewed  that 
"more"  information,  and  a  mark  of  "o"  indicated  that  the  participant 
did  not  view  more  information.  If  no  "more"  option  was  available,  a 
dash  (-)  was  recorded  beside  the  hash  mark  in  the  witness  box. 

•  Informant.  One  point  was  recorded  each  time  the  participant  went  to 
the  informant  for  clues.  If  the  participant  had  the  option  to  obtain 
more  information  while  viewing  the  informant,  an  additional  mark  of 
"m"  indicated  that  the  participant  viewed  more  information,  and  a 
mark  of  "o"  indicated  that  the  participant  did  not  view  more 
information.  If  no  "more"  option  was  available,  a  dash  (-)  was 
recorded  beside  the  hash  mark  in  the  informant  box. 

•  Scanner.  One  point  was  recorded  each  time  the  participant  went  to  the 
scanner  for  clues.  An  option  to  scan  further  was  come  across 
randomly  after  an  initial  scan  was  completed.  An  "f"  was  recorded 
beside  the  scan  hash  mark  if  the  scan  further  option  was  taken. 

•  Note  Taking.  One  point  was  recorded  each  time  the  participant  wrote 
a  note.  For  example,  if  the  participant  wrote  a  note,  the  performed 
another  recordable  activity,  such  as  using  the  encyclopedia,  then  wrote 
another  note,  two  points  would  be  recorded.  On  the  other  hand,  if  the 
participant  wrote  a  note,  thought  for  a  while,  then  wrote  another  note, 
only  one  point  would  be  recorded  since  the  participant  did  not 
perform  a  recordable  activity  between  note  writing. 

•  Reference.  One  point  was  recorded  each  time  the  participant  used  the 
New  American  Desktop  Encyclopedia  to  look  up  Information.  For 
example,  if  the  participant  used  the  encyclopedia,  then  performed 
another  recordable  activity,  such  as  taking  a  note,  then  referred  back  to 
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the  encyclopedia,  two  points  would  be  recorded  for  this  measure.  On 
the  other  hand,  if  the  participant  used  the  encyclopedia,  thought  for  a 
while,  then  used  the  encyclopedia  again,  only  one  point  would  be 
recorded  since  the  participant  did  not  perform  a  recordable  activity 
between  the  two  instances  of  using  the  encyclopedia. 
Program  Documentation.  One  point  was  recorded  each  time  the 
participant  used  the  program  documentation  to  look  up  information. 
For  example,  if  the  participant  used  the  program  documentation,  then 
performed  another  recordable  activity,  such  as  taking  a  note,  then 
referred  back  to  the  program  documentation,  two  points  would  be 
recordedfor  this  measure.  On  the  other  hand,  if  the  participant  used 
the  program  documentation,  thought  for  a  while,  then  used  the 
program  documentation  again,  only  one  point  would  be  recorded  since 
the  participant  did  not  perform  a  recordable  activity  between  the  two 
instances  of  using  the  program  documentation. 
Select  Clues.  One  point  was  recorded  for  each  plain  clue,  such  as  sex, 
eye  or  hair  color,  the  participant  entered  into  the  crime  computer 
database. 

Subtle  Clues.  One  point  was  recorded  for  each  subtle  clue,  such  as  the 
criminal's  favorite  author  or  artist,  the  participant  entered  into  the 
crime  computer  database.  Randomly,  the  game  would  present  hair  or 
eye  colors  which  required  some  inferential  problem  solving  skill.  For 
example,  "she  had  hair  the  color  of  mahogany"  is  an  inferred  hair  color 
clue.  When  such  a  clue  was  correctly  recorded  in  the  crime  computer 
database,  one  point  was  recorded  in  the  inferred  clue  box. 
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•  Arrest  Warrant  Attempt.  One  point  was  recorded  each  time  the 
participant  attempted  to  get  an  arrest  warrant  by  clicking  the  Compute 
button  in  the  crime  computer  database. 

•  Arrest  Warrant  One  point  was  recorded  when  the  participant 
obtained  an  arrest  warrant. 

•  Task  Complete.  One  point  was  recorded  when  the  participant 
completed  the  task  by  arresting  the  suspected  thief.  This  means  that  if 
the  participant  falsely  arrested  a  thief,  the  task  was  recorded  as 
complete  even  though  it  was  not  successfully  completed. 

•  False  Arrest.  One  point  was  recorded  if  the  participant  falsely  arrested 
a  suspect,  by  either  not  having  an  arrest  warrant  or  having  an  arrest 
warrant  for  the  wrong  criminal. 

•  False  Leads.  One  point  was  recorded  if  the  participant  followed  the 
suspect  to  an  incorrect  destination. 

Success  in  completing  the  task,  which  we  call  outcome,  was  operationalized 
in  the  following  manner.  We  scored  participants  who  acquired  the  correct  arrest 
warrant,  successfully  tracked  the  thief,  and  made  the  arrest  as  3.  Participants 
who  acquired  the  correct  arrest  warrant  or  tracked  the  thief  and  made  the  arrest 
were  2.  Participants  who  neither  obtained  the  correct  arrest  warrant  nor  tracked 
the  thief  and  made  the  arrest  were  1. 

We  determined  the  effectiveness  in  completing  the  task  by  scoring  and 
combining  the  four  key  task  strategies,  witness,  informant,  scanner,  and  notes, 
into  one  measure,  called  effectiveness.  We  judged  a  participant  to  be  effective  in 
their  use  of  key  strategies  if  they  don't  use  the  witness  and  informant  more  than 
once  for  each  location,  don't  use  the  scanner  at  all,  and  take  at  least  one  note  at 
each  location. 
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Scoring  each  of  these  strategies  was  in  the  following  manner.  If  the 
participant  accessed  the  witness  clue  one  or  fewer  times  per  location,  we 
awarded  one  point.  Accessing  the  witness  clue  two  or  more  times  resulted  in  no 
points.  We  totaled  the  points,  then  divided  by  the  total  number  of  locations 
visited  by  the  participant,  resulting  in  a  score  between  0  and  1.  We  used  the 
same  methodology  to  calculate  the  score  for  the  informant.  Scoring  for  the 
scanner  was  similar  to  the  witness  and  informant,  however  not  using  the  scanner 
resulted  in  one  point  per  location,  while  using  the  scanner  at  a  location  resulted 
in  no  points.  Scoring  notes  was  also  similar  to  the  witness  and  scanner,  with  one 
point  awarded  per  location  if  there  was  at  least  one  note,  and  no  points  awarded 
if  there  were  no  notes.  We  calculated  a  mean  score  for  the  effectiveness  of  the 
key  strategies  by  adding  the  scores  for  witness,  informant,  scanner,  and  notes, 
then  dividing  by  four. 

Two  raters  judged  the  articulations.  They  classified  the  articulations  into  four 
equal  sets  of  bad,  good,  better  and  best  articulations.  We  then  combined  these 
articulations  into  two  equal  sets,  with  the  bad  and  good  sets  classified  as  weak 
articulations,  and  the  better  and  best  sets  as  strong  articulations  This  rating 
resulted  in  an  inter-rater  reliability  of  .75.  Differences  were  discussed  and 
resolved  by  the  two  raters. 

Debrief.  At  the  conclusion  of  the  experiment,  the  experimenter  explained  to 
the  participant  the  actual  purpose  of  the  study.  In  addition,  the  experimenter 
was  able  to  ask  the  participant  several  informal  qualitative  questions  about  his  or 
her  performance.  We  wrote  the  responses  to  these  questions  on  the  participants 
data  sheet.  Participants  were  also  able  to  ask  the  experimenter  questions  about 
the  study.  We  concluded  the  study  by  thanking  the  participant  for  their  help  and 
assuring  them  they  would  receive  extra  credit  for  their  class. 
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Results  and  Discussion 
Hypotheses  One  and  Two 

Our  first  hypothesis  predicted  that  both  the  novice  and  expert  modeling 
would  result  in  better  task  performance  than  the  control.  We  determined  task 
performance  through  two  measures,  outcome  and  chronotime.  Participants  who 
obtained  an  arrest  warrant  and  tracked  the  thief  successfully  received  credit  the 
remaining  chronotime  WHICSD.  This  number  ranged  from  1  to  21,  whereby  the 
higher  the  number,  the  more  successful  we  judged  the  participant.  All  other 
participants  received  0  chronotime  illustrating  their  failure  in  successfully 
completing  the  game. 

We  used  a  one-factor  ANOVA  to  analyze  these  variables.  There  was  no 
significant  difference  between  the  modeling  and  control  treatments  for  both  the 
outcome  and  chronotime  variables.  Thus,  there  was  no  support  for  hypothesis 
one.  This  finding  contradicts  the  findings  of  many  of  the  modeling  studies 
referenced  in  this  paper,  whereby  modeling  increased  a  participant's  task 
performance. 

There  are  a  number  of  possible  explanations  for  why  we  did  not  observe 
significant  differences  between  the  expert,  novice  and  control  treatments.  First, 
the  expert  and  novice  modeling  may  have  contributed  to  a  false  sense  of 
expertise  among  participants.  That  is,  by  watching  the  modeling  movie, 
participants  became  overconfident  in  their  ability  to  play  the  game.  To  further 
support  this  explanation,  data  from  the  debriefing  shows  that  six  participants 
mentioned  that  the  modeling  movie  contributed  to  them  ignoring  the  program 
documentation.  As  one  participant  said,  "I  forgot  about  the  instructions  after 
watching  the  movie." 

A  second  explanation  suggests  an  opposite  phenomena  for  the  control. 
Instead  of  having  a  false  sense  of  expertise  and  being  overconfident  in  their 
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ability  to  solve  the  task,  the  control  group  struggled  with  the  problem.  Thus, 
they  may  have  had  to  use  a  greater  set  of  problem  solving  heuristics  and  learn 
more  from  their  trial  and  error.  Although  the  finding  was  not  significant,  the 
control  took  longer  (M=1121  seconds)  to  complete  the  task  than  either  the  expert 
(M=795  seconds)  or  the  novice  (M=854  seconds).  This  may  indicate  that  the 
control  was  thinking  more  about  the  task  than  either  the  expert  or  novice. 

A  third  explanation  is  the  nature  of  the  task,  a  computer  game.  While  none  of 
the  participants  had  ever  played  WITICSD,  a  number  of  participants  indicated 
that  they  had  played  other  computer  or  video  games  before.  Thus,  the  successful 
performance  of  the  control  group  may  be  due  to  the  skills  and  strategies 
participants  developed  while  playing  other  computer  games. 

A  fourth  explanation  is  orienting  activities.  Perhaps  the  orientation  and 
instructions  given  to  each  participant  revealed  too  much  game  strategy, 
especially  for  the  control  group. 

We  found  no  support  for  our  second  hypothesis  either.  Our  second 
hypothesis  predicted  that  novice  modeling  would  result  in  better  task 
performance  than  expert  modeling.  For  this  analysis,  we  used  the  same  outcome 
and  chronotime  dependent  variables  as  in  hypothesis  one.  A  one-factor  ANOVA 
analyzed  the  variables.  There  was  no  significant  difference  between  novice  and 
expert  modeling. 

The  lack  of  difference  between  the  novice  and  expert  modeling  may  be 
caused  by  the  novice  group  not  observing  successful  task  completion.  We 
anticipated  that  the  novice  modeling  would  result  in  stronger  performance  since 
the  participants  would  learn  to  avoid  the  common  mistakes  novices  tend  to 
make.  However,  the  novice  in  the  novice  modeling  movie  did  not  successfully 
complete  the  task,  whereas  the  expert  in  the  expert  modeling  movie  did.  Thus, 
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participants  in  the  novice  group  may  not  have  had  a  clear  idea  of  how  to 
recognize  successful  task  performance. 
Hypotheses  Three  and  Four 

Hypothesis  three  predicted  that  expert  and  novice  modeling  would  result  in 
better  use  of  key  task  strategies  than  the  control,  and  hypothesis  four  predicted 
novice  modeling  would  result  better  use  of  key  task  strategies  than  expert 
modeling.  As  defined  above,  we  created  a  dependent  measure  called 
effectiveness  by  scoring  the  participant's  use  of  witness,  informant,  scanner,  and 
note  taking.  Thus,  we  compared  expert  and  novice  modeling  to  the  control  by 
examining  the  effectiveness.  A  one-factor  ANOVA  analyzed  the  variables. 

A  significant  difference  (Df  2:42,  F=6.32,  p<.01)  was  found  between  the  expert 
(M=.809),  novice  (M=.841)  and  control  (M=.693)  treatments.  Additionally,  a 
Fisher  PLSD  test  revealed  that  novice  was  significantly  different  from  control 
(.088,  p<.05)  and  that  expert  was  significantly  different  from  control  (.091,  p<.05). 
Thus,  hypothesis  three  has  support.  While  hypothesis  four  appears  supported, 
with  novice  modeling  having  a  slightly  higher  mean  than  the  expert  modeling,  a 
follow-up  Fisher  PLSD  test  between  expert  and  novice  yielded  no  significance. 

Although  we  found  no  significance  with  hypothesis  one  and  two,  the 
significance  found  in  hypothesis  three  identifies  a  key  effect  of  the  expert  and 
novice  modeling.  From  the  results,  we  can  infer  that  participants  who  viewed  a 
model  learned  to  use  the  key  task  strategies  better  than  the  control.  Additionally, 
there  is  slight  evidence  that  the  novice  group  learned  to  use  the  key  task 
strategies  better  than  the  expert  group. 

One  would  expect  that  if  participants  used  the  key  task  strategies  effectively, 
they  would  successfully  complete  the  game.  However,  the  data  show  an 
incongruence  between  hypothesis  one  and  three,  and  hypothesis  two  and  four.  It 
is  possible  to  explain  this  incongruence  because  while  participants  in  the  expert 
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and  novice  modeling  treatment  groups  may  have  used  key  strategies  more 
effectively  than  the  control  group,  we  have  no  direct  way  of  knowing  how  they 
interpreted  the  information  gained  from  exercising  those  strategies. 
Interpretation  can  only  be  inferred  by  a  participant's  successful  completion  of  the 
game.  If  a  participant  successfully  completed  the  game,  then  we  can  infer  that 
they  were  successful  in  interpreting  the  clues.  The  key  task  strategies  are  only  a 
means  of  uncovering  clues  necessary  for  successful  completion.  Thus,  they  have 
only  a  partial  role  in  predicting  successful  completion. 
Hypotheses  Six  through  Eight 

Our  fifth  hypothesis  predicted  that  novice  and  expert  modeling  would  result 
in  stronger  articulations  than  the  control.  We  used  a  one-factor  ANOVA  to 
analyze  the  variables.  We  found  no  significant  difference,  indicating  that  neither 
novice  nor  expert  modeling  resulted  in  stronger  articulations  by  the  participant. 
Hypothesis  five  had  no  support. 

Hypothesis  six  predicted  that  novice  modeling  would  result  in  stronger 
articulations  than  expert  modeling.  A  one-factor  ANOVA  analyzed  the 
variables.  There  was  no  significant  difference,  thus  the  novice  model  did  not 
cause  stronger  articulations  than  the  expert  model.  We  did  not  support 
hypothesis  six. 

Our  seventh  hypothesis  was  similar  to  the  third  hypothesis  except  we 
predicted  that  participants  with  strong  articulations  would  use  key  task 
strategies  more  effectively  than  those  with  weak  articulations.  Independent 
raters  scored  the  quality  of  articulations  as  either  good  or  bad.  Thus,  we 
compared  the  good  and  bad  articulations  with  effectiveness  using  a  one-factor 
ANOVA.  There  was  no  significant  difference,  indicating  that  the  quality  of 
articulations  did  not  affect  the  use  of  key  strategies.  Hypothesis  seven  had  no 
support. 
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The  explanation  of  why  there  was  no  difference  for  this  hypothesis  is  similar 
to  the  explanations  presented  for  hypotheses  one  and  two.  It  is  possible  that  the 
novice  and  expert  modeling  caused  a  false  sense  of  expertise  in  participants, 
resulting  in  weak  articulations  for  some.  Additionally,  the  orientation  activities 
may  have  provided  too  much  information  about  how  to  play  the  game. 

Hypothesis  eight  predicted  that  strong  articulations  would  result  in  better 
task  performance  than  weak  articulations.  For  this  analysis,  we  used  the 
outcome  and  chronotime  dependent  variables  used  in  hypothesis  one  and  two  as 
a  measure  of  task  performance.  We  also  used  the  same  articulation  ratings  found 
in  hypothesis  six  and  seven.  A  one-factor  ANOVA  analyzed  the  variables. 

The  results  of  this  analysis  were  significant  for  both  outcome  (Df  1:43, 
F=4.648,  p<.01)  and  chronotime  (Df  1:43,  F=10.078,  p<.01).  A  Fisher  PLSD  test 
(.449,  p<.05)  revealed  that  participants  with  articulations  classified  as  "good"  had 
a  mean  outcome  score  of  2.435  while  "bad"  articulations  had  a  mean  outcome 
score  of  1.727.  A  Fisher  PLSD  test  for  chronotime  (3.908,  p<.05)  showed  that  the 
mean  chronotime  for  "good"  articulation  was  6.087  while  mean  chronotime  for 
"bad"  articulation  was  1.909.  On  the  basis  of  these  results,  there  was  support  for 
hypothesis  eight  -  strong  articulations  contribute  to  successful  task  performance. 

This  finding  shows  the  importance  of  designing  instructional  interventions 
that  contribute  to  forming  strong  articulations,  since  strong  articulations  are  a 
good  predictor  of  success.  We  should  consider  designing  instructional 
interventions  with  the  goal  of  enabling  the  learner  to  form  strong  articulations. 
This  strategy  contradicts  the  traditional  concern  for  the  terminal  goal  - 
successful  performance  on  a  task.  Facilitating  successful  performance  was  our 
paradigm  when  we  designed  the  expert  and  novice  modeling  movies.  An 
articulation-focused  design  strategy  changes  our  paradigm  to  where  we  are  more 
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concerned  with  enabling  learners  to  talk  about  how  they  solve  problems  rather 
than  strictly  focusing  on  enabling  them  to  complete  the  task  successfully. 

Summary  and  Conclusion 

The  results  of  this  study  offer  a  number  of  interesting  findings.  We  were 
unable  to  replicate  the  results  found  in  previous  modeling  studies  in  which 
modeling  enabled  participants  to  perform  better  on  a  task.  We  also  found  no 
difference  between  the  novice  and  expert  groups  in  terms  of  task  performance. 
However,  we  did  find  that  novice  and  expert  groups  used  key  task  strategies 
more  effectively  than  the  control  group,  and  that  there  may  have  been  only  a 
slight  difference  between  the  novice  and  expert  groups. 

Our  articulation  task  yielded  similar  results.  We  found  no  differences  for  the 
expert,  novice  and  control  treatments  on  the  quality  of  the  articulations. 
Additionally,  participants  with  good  articulations  did  not  use  key  task  strategies 
any  differently  from  participants  with  bad  articulations.  However,  participants 
with  good  articulations  performed  better  on  the  task  than  participants  with  bad 
articulations. 

Why  didn't  our  findings  support  our  many  of  our  key  hypotheses?  We 
believe  the  main  reason  is  the  effect  of  the  orienting  activities.  In  order  to  make 
sure  participants  were  technically  able  to  use  WITICSD,  we  conducted  an 
extensive  set  of  orienting  activities  common  to  all  treatment  groups.  While  we 
accomplished  our  goal  of  enabling  participants  to  be  technically  competent,  we 
may  have  told  them  too  much.  This  might  account  for  the  lack  of  difference  in 
task  performance  and  articulation  quality  between  all  treatment  groups. 

There  are  two  key  implications  from  our  study.  First,  cognitive  modeling, 
regardless  of  the  verbalization  technique  used,  aids  learners  in  adopting  key  task 
strategies.  Thus,  one  should  adopt  cognitive  modeling  as  an  instructional 
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strategy  if  the  instructional  objectives  require  the  learning  of  heuristics.  Second, 
since  the  quality  of  articulation  is  a  predictor  of  task  performance,  we  should 
design  instruction  to  achieve  quality  articulations.  Learners  who  can  explain 
how  they  will  solve  a  problem  will  experience  more  successful  task  performance. 

Future  research  in  the  area  of  cognitive  modeling  should  continue  to  explore 
variations  in  verbalization  and  their  effects.  Researchers  should  take  time  to 
consider  the  orienting  activities,  especially  when  working  in  a  computer-based 
training  environment.  Additional  research  should  discover  how  to  facilitate 
quality  articulations  in  instructional  programs. 
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Appendix  A  -  Instructions  Transcript 

You  will  be  playing  a  computer  game  called  Where  In  Time  Is  Carmen  San  Diego.  The  object 
of  the  game  is  to  catch  a  thief  who  has  stolen  a  historical  treasure. 

In  order  to  catch  the  thief,  you  must  do  three  things: 

1)  You  must  gather  clues  to  determine  the  thief  s  identify  and  to  follow  the  thief  s  trail. 
Clues  are  gathered  by  using  the  search  button,  followed  by  either  the  Witness,  Informant  or 
Scanner  buttons.  The  Witness  and  Informant  buttons  may  contain  clues  about  the  thief  s  identity 
and  his  or  her  next  destination.  The  Scanner  button  contains  clues  only  about  the  thief  s  next 
destination. 

2)  You  must  follow  the  thief  s  trail  from  country  to  country.  The  Travel  button  enables  you 
to  travel  to  the  next  destination.  Use  the  clues  youVe  gathered  to  determine  the  next  destination. 

3)  You  must  obtain  an  arrest  warrant  before  arresting  the  thief.  Arrest  warrants  can  be 
obtained  through  the  Data,  then  Evidence  buttons.  Record  the  clues  you've  gathered  and  use  the 
Compute  button  to  determine  if  the  clues  match  a  criminal  and  enable  you  to  obtain  an  arrest 
warrant. 

Here  are  some  hints  for  playing  the  game: 

1)  It's  a  good  idea  to  keep  notes  on  paper  as  you  gather  clues. 

2)  Remember  to  work  quickly  and  efficiently  -  you  only  have  a  certai  n  amount  of  time  to 
complete  the  task.  Time  is  subtracted  pnly  when  you  a)  travel  from  one  location  to  another  (here 
the  time  used  is  approximately  3-5  hours),  b)  obtain  clues  from  the  witness,  informant  or  crime 
scene  scan  (here  the  time  used  is  approximately  1  to  2  hours),  or  c)  compute  the  evidence  to  try 
and  obtain  an  arrest  warrant  (here  the  time  used  is  1-2  hours).  No  time  is  counted  against  you 
when  you  look  up  information  from  the  reference  sources. 

3)  You'll  know  when  you  have  tracked  the  thief  successfully  when  you  see  suspicious 
persons.  The  suspect  you're  tracking  uses  other  gang  members  to  check  you  out  when  your  are 
getting  dose.  If  you  interrogate  a  witness,  informant  or  crime  scene  and  they  know  nothing 
you're  probably  in  the  wrong  location.  You'll  then  have  to  backtrack  to  the  previous  location  to 
get  back  on  the  trail  of  the  criminal. 

4)  When  the  criminal  is  at  the  same  location  your  are,  he  or  she  launches  a  sneak  attack 
against  you.  Make  sure  you  have  your  warrant  when  this  happens  or  else  the  criminal  will  get 
away.  If  you  have  your  warrant,  look  to  other  sources  of  clues  to  find  the  criminal. 

5)  The  key  to  tracking  the  criminal  is  deciphering  the  clues  you  find.  You  can  decipher 
clues  with  the  New  American  Desk  Encyclopedia.  Be  sure  to  look  at  the  'How  To  Use"  section  in 
the  Encyclopedia. 

Here's  an  example  of  how  to  acquire  clues  and  evidence: 

For  example,  let's  say  you  use  the  Witness  button  and  the  clue  is,  "She  wanted  to  talk  with 
Dag  Hammarskjold  about  economy."  You  could  then  look  up  "Hammarskjold"  in  the 
encyclopedia  and  find  out  he  was  a  Swedish  stateman  in  the  1900's.  This  clue  would  tell  you  the 
criminal's  next  destination  is  probably  Sweden  in  the  1900's. 

In  addition  to  this  clue,  there  is  another  button  called  More.  You  can  click  this  button  to  get 
evidence  about  the  criminals  identity.  For  example,  if  the  clue  is  "She  just  finished  reading  the 
book  'Animal  Farm'.",  you  would  go  to  the  evidence  selector  by  clicking  the  data  button.  You 
would  then  select  each  author,  and  look  up  in  the  encyclopedia  if  that  author  wrote  'Animal 
Farm*.  So  in  this  case,  we  would  find  George  Orwell  and  keep  him  selected  as  a  piece  of 
evidence. 
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Appendix  B  -  Transcription  of  Expert  Movie 

This  movie  is  a  demonstration  on  how  to  play  Where  in  Time  is  Carmen  San  Diego.  My  name 
is  Peter  and  111  be  showing  you  how  to  play  the  game.  In  this  case  the  Indian  Constitution  has 
been  stolen  from  India  in  1947  by  an  unknown  male.  I  have  written  down  these  facts  in  my 
notebook.  The  Federal  Time  Commission  has  given  us  33  hours  to  solve  the  crime.  Well  begin 
our  investigation  by  searching  for  dues  at  the  scene  of  the  crime.  HI  first  investigate  the  witness. 
Here  I  find  that  the  thief  wanted  to  join  a  group  called  the  Samurai.  This  gives  me  some 
information  as  to  the  next  destination  of  the  thief.  Samurai,  if  I  look  'em  up  in  the  Desktop 
Encyclopedia,  are  from  Japan  during  the  years  1000-1600  A.D.  Ill  write  this  down  in  my  notepad 
for  future  reference.  The  'More*  button  gives  you  clues  as  to  the  identity  of  the  thief.  Here  (the 
more  button)  I  get  the  clue  that  the  thief  had  hazel  eyes,  and  111  write  that  down  on  my  notepad 
as  well.  When  1  uncover  data  about  the  thief  s  identity  I'll  want  to  record  them  in  my  evidence 
folder.  I  look  back  in  my  notes  and  I  see  that  one  of  the  clues  is  that  the  thief  is  a  male.  Ill  enter 
male  under  the  sex  option.  I  also  know  that  the  thief  s  eyes  are  hazel,  so  I'll  click  the  eyes  button 
until  hazel  appears.  I  won't  compute  yet  to  get  an  arrest  warrant  because  I  don't  have  enough 
information  and  it  would  just  waste  valuable  Chronotime.  so  I'll  continue  my  search. 

The  next  place  III  investigate  is  the  informant.  Here  I  learn  the  thief  wanted  to  spy  on  the 
Anu  tribe.  Once  again  this  clue  will  give  me  some  idea  as  to  the  location  the  thief  went  to  next. 
And  if  I  look  up  the  Anu  tribe  in  the  Desktop  Encyclopedia,  111  find  that  the  Anu  tribe  was  part  of 
Japan  as  well,  so  1  can  deduce  from  my  Samurai  clue  and  this  clue  that  the  next  destination  of  the 
thief  is  Japan.  When  I  click  the  Travel'  button,  I'll  see  that  Japan  is  one  of  the  destinations,  and 
that's  where  111  follow  the  thief  to  next  Notice  that  I  did  not  go  to  the  'Scanner'  for  any  clues.  This 
would  be  a  waste  of  Chronotime  since  the  'Scanner'  only  gives  me  dues  as  to  the  thief  s  next 
destination,  and  I  was  already  able  to  deduce  the  thief  s  next  destination  based  on  the  two  clues  I 
received  previously. 

As  in  the  previous  location,  I'll  start  my  search  by  going  to  the  Witness.  O.K.,  a  V.I.L.E. 
henchman  has  been  detected.  This  is  feedback  the  game  gives  me  to  tell  me  I  have  correctly 
followed  the  thief  to  the  next  destination.  If  I  hadn't  followed  the  thief  to  the  next  destination,  the 
witness  would  have  said  to  me,  he  had  never  heard  of  that  person,  or  doesn't  know  anything 
about  what  I'm  talking  about.  The  witness  tells  me  that  the  thief  wanted  to  go  to  Lima,  so  I  write 
this  down  on  my  notepad,  and  I  look  up  Lima  in  my  Desktop  Encyclopedia,  and  find  out  that 
Lima  is  the  Capital  of  Peru.  This  gives  me  a  pretty  good  idea  that  the  next  destination  of  the  thtef 
is  Peru. 

I'll  click  the  'More'  button  to  find  out  another  due  as  to  the  thief  s  identity.  Here  I  find  that  the 
thief  was  holding  a  copy  of  The  AHunchback  of  Notre  Dame.  This  gives  me  some  idea  as  to  the 
author  the  thief  liked.  I'll  immediately  record  this  clue  on  my  notepad  as  well  as  in  my  evidence 
folder.  Ill  click  the  'Author*  button  and  the  first  author  that  appears  is  Doysteyevski.  I'll  look  up 
Doysteyevski  in  the  Desktop  Encydopedia  and  see  if  he  had  anything  to  do  with  a  book  about  the 
Hunchback  of  Notre  Dame.  He  doesn't,  so  111  have  to  go  on  to  the  next  author.  The  next  author  is 
Victor  Hugo.  I  look  him  up  in  the  Desktop  Encydopedia  and  sure  enough  he  is  the  author  of  The 
Hunchback  of  Notre  Dame.  Since  I  have  three  clues,  this  is  usually  enough  to  compute  an  arrest 
warrant,  and  sure  enough  the  clues  match  a  particular  suspect 

Now  that  I  have  an  arrest  warrant  for  the  thief,  I  have  to  track  him  as  quick  as  possible  to  the 
next  destination.  From  the  due  the  witness  gave  me,  I  know  his  next  destination  is  probably  Peru. 
So  1  don  t  need  any  more  clues,  and  111  travel  directly  to  that  location.  Once  again,  I'll  begin  by 
searching  for  clues...ahh,  I've  found  another  V.I.L.E.  henchman  this  means  I  still  must  be  on  the 
right  track  of  the  thief.  The  clue  the  witness  gives  me  is  that  the  thief  wanted  to  meet  the  founder 
of  the  Capetians.  I  write  Capetians  down  in  my  notepad,  and  look  it  up  in  my  Desktop 
Encyclopedia.  The  Capetians  are  from  France,  so  I  can  be  pretty  sure  that  the  next  destination  of 
the  thief  is  France.  Since  I'm  pretty  sure  of  that  clue,  I  won't  waste  valuable  time  searching  for 
other  clues;  I'll  go  directly  to  the  thief  s  next  destination. 
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Til  start  once  again  by  searching  for  clues  at  the  witness.  It  looks  like  the  thief  is  trying  to  kill 
me.  This  means  he  must  be  at  one  of  the  other  sources  of  information  at  this  destination,  It  also 
means  that  if  I  don't  have  my  arrest  warrant  I'd  better  go  get  it.  Otherwise,  Til  be  charged  with  a 
false  arrest.  But  since  I  have  my  arrest  warrant,  I  can  go  searching  for  the  thief  to  try  and  catch 
him.  Let's  try  the  scanner  to  see  if  that  will  locate  the  thief.  There  he  is,  now  the  capture  robot  will 
go  do  the  work.  Good,  we've  captured  the  thief.  We'll  now  send  the  data  to  the  crime  computer  to 
verify  his  identity,  and  we'll  get  a  congratulatory  message  from  the  Chief.  That's  how  you  play 
the  game  Where  in  Time  is  Carmen  San  Diego.  Remember  to  record  notes  of  clues  you  uncover 
during  your  search,  and  to  use  the  Desktop  Encyclopedia  to  interpret  those  clues  and  determine 
the  next  destination  or  ideitity  of  the  thief.  Also  remember  to  get  an  arrest  warrant  before 
capturing  the  thief,  now  be  given  a  chance  to  play  the  game  yourself.  Good  luck. 
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Appendix  C  -  Transcription  of  Novice  Movie 

"In  this  demonstration  we'll  be  watching  a  first-time  player  play  the  game  Where  in  Time  is 
Carmen  San  Diego?  My  name  is  Peter  and  I'll  be  commenting  on  the  player's  performance  as  we 
watch  the  game.  In  this  case  an  unknown  male  has  stolen  the  Indian  Constitution  from  India  in 
1947.  The  player  has  proceeded  to  the  scene  of  the  crime,  and  has  thirty-three  hours  in  which  to 
solve  the  case.  Let's  watch  as  the  player  begins  her  investigation. 

Here  we  see  the  player  searching  for  clues  at  the  witness.  This  particular  clue  about  a  gang  of 
Samurai  gives  the  player  a  tip  about  where  the  thief  went  to  next.  However,  the  player  does  not 
write  down  the  clue  or  does  not  look  at  the  Desktop  Encyclopedia  in  order  to  decipher  the  clue. 
While  the  player  was  at  the  witness,  she  missed  clicking  on  the  'More'  button.  The  'More'  button 
would  have  given  her  a  clue  as  to  the  identity  of  the  thief.  This  would  have  helped  her  to  get  an 
arrest  warrant.  But  now  we  see  that  she  wants  to  go  travel  somewhere. 

Without  deciphering  the  clues  the  player  received  at  the  witness,  regarding  the  Samurai 
gang,  she  has  travelled  randomly  to  the  United  States.  Let's  see  what  happens  now.  As  we  can  see 
the  witness  has  nothing  the  tell  the  player.  This  means  the  player  has  travelled  to  the  wrong 
location.  What  she  needs  to  do  now  is  get  back  to  her  previous  location  and  get  back  on  this  trail 
of  the  thief.  The  player  has  now  returned  to  India  in  the  1900's,  therefore  she  is  back  on  the  trail  of 
the  thief.  Notice  the  Chronotime  is  at  27  hours.  Each  time  the  player  travels  to  a  destination,  or 
obtains  a  clue  from  the  witness,  informant,  or  scanner,  time  is  counted  off  the  total  chronotime. 
This  means  you  have  to  be  sure  of  where  you're  travelling,  or  to  be  efficient  in  searching  for  your 
clues  so  you  do  not  run  out  of  chronotime.  Running  out  of  chronotime  means  you  will  not  be  able 
to  catch  the  thief.  Let's  watch  now  as  she  continues  her  search  . 

Here  the  player  gets  the  clue  of  the  Shinto  Shrine.  She  writes  it  down  on  her  notepad  and 
looks  up  'Shinto*  in  her  Desktop  Encyclopedia.  Once  again  the  player  forgets  to  click  the  'More1 
button  under  the  witness  to  get  a  clue  as  to  the  identity  of  the  thief. 

In  the  informant,  the  player  finds  the  clue  of  the  Golden  Hall  of  Haruji.  She  writes  these 
words  down  in  her  notepad,  and  looks  up  both  Golden  Hall  and  Haruji  in  the  Desktop 
Encyclopedia.  She  finds  nothing  for  these  terms.  Based  on  the  clues  she  received  from  the  witness 
and  the  informant,  the  player  seems  to  have  deduced  that  the  next  destination  of  the  thief  is 
Japan.  What  we  have  just  witnessed  is  the  appearance  of  a  V.I.L.E.  henchman.  These  V.I.L.E. 
henchmen  check  up  on  you  when  you  are  following  the  thief  correctly.  Thus,  the  player  has 
correctly  followed  the  thief  to  Japan.  In  the  informant  the  player  gets  the  clue  of  Austin  &  Emma. 
This  clue  gives  the  player  some  idea  of  where  the  thief  travelled  to  next.  She  writes  the  clue  down 
on  her  notepad,  and  looks  up  Austin  in  the  Desk  Encyclopedia.  Once  again  the  player  missed 
clicking  on  the  'More*  button  to  get  a  clue  as  to  the  identity  of  the  thief.  In  the  witness,  the  player 
gets  the  clue  of  Rugby.  She  writes  Rugby  down  on  her  notepad  and  looks  up  Rugby  in  the  Desk 
Encyclopedia,  however,  cannot  find  an  entry  for  it.  Once  again,  the  player  has  not  clicked  on  the 
'More'  button  in  order  to  get  a  clue  as  to  the  identity  the  thief.  Without  these  types  of  clues,  the 
player  will  not  be  able  get  an  arrest  warrant,  thus,  the  player  will  not  be  able  to  win  the  game.  It's 
very  important  that,  when  you  do  not  have  an  arrest  warrant,  you  always  click  the  'More'  button 
to  get  some  idea  of  who  the  thief  is. 

It  seems  that  from  the  clues  of  Austin  &  Emma  and  of  Rugby  the  player  was  able  to  deduct? 
that  the  next  location  that  the  thief  travelled  to  was  England.  Let's  see  if  that  turns  out  to  be 
correct  Aonce  again  the  V.I.L.E.  henchman  has  appeared,  so  the  player  has  successfully  tracked 
the  thief  to  the  next  destination.  Here  (under  button  which  has  just  been  pushed  and  illustrated 
on  the  screen)  the  player  receives  the  clue  of  a  mousetrap.  She  writes  it  down  on  her  notepad  and 
looks  it  up  in  the  Desk  Encyclopedia,  but  finds  nothing.  I  tend  to  Afeel  that  the  scanner  is  a  waste 
of  time,  most  of  the  clues  that  you  need  to  Afollow  the  thief  can  be  found  in  the  witness  and 
informant.  Plus,  the  scanner  will  never  have  a  clue  as  to  the  identity  of  the  thief.  1  usually  only  the 
use  the  scanner  as  a  last  resort  if  I  cannot  figure  out  the  destination  of  the  thief  from  tho  clues  1 
obtain  in  the  witness  and  informant.  In  the  witness  the  player  gets  the  clue  Titanic.  She  writes  it 
down  on  her  notepad,  and  looks  it  up  in  the  Desk  Encyclopedia  and  finds  a  reference  for  it. 
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Based  on  the  clue  of  the  Titanic,  the  player  has  deduced  that  the  next  destination  of  the  thief 
is  England  in  the  1900's.  Note  that  back  in  the  witness,  she  did  not  dick  the  'More*  button  once 
again.  This  denied  her  of  an  important  clue  as  to  the  identity  of  the  thief,  which  would  have 
helped  her  get  an  arrest  warrant  When  a  life-  threatening  trap  has  been  set  against  the  player,  as 
the  one  that  we  just  saw,  this  means  that  the  thief  is  somewhere  at  this  destination  and  that  the 
player  has  to  try  to  find  the  thief  s  whereabouts  at  either  the  witness,  informant  or  scanner.  It 
appears  that  the  player  has  finally  realized  that  she  needs  an  arrest  warrant  to  catch  the  thief. 
Since  the  player  did  not  investigate  any  of  the  'More*  buttons  to  obtain  clues  as  to  the  identity  of 
the  thief,  the  only  clue  she  has  is  that  the  thief  is  a  male.  Usually-  you  need  three  or  more  clues  in 
order  to  obtain  an  arrest  warrant  So  I  do  not  think  this  player  will  be  getting  an  arrest  warrant 
this  time.  As  I  mentioned,  only  one  clue  is  insufficient  data,  and  returns  a  whole  list  of  possible 
suspects.  Thus  an  arrest  warrant  cannot  be  issued. 

Another  trap  has  been  uncovered,  the  player  is  now  getting  close  to  capturing  the  thief.  The 
player  has  located  the  thief,  and  now  the  capture  robot  will  be  sent  to  make  the  arrest.  Well,  the 
player  has  successfully  tracked  Russ  T.  Hinge  but  unfortunately,  without  *n  arrest  warrant,  Russ 
T.  Hinge  is  allowed  to  go  free.  Although  the  player  made  a  minor  mistake  at  the  beginning  of  the 
game,  she  was  able  to  successfully  track  the  thief  from  destination  ~.  destination.  However,  she 
did  forget  to  acquire  clues  as  to  the  identity  of  the  thief  which  prevented  her  from  getting  an 
arrest  warrant.  You'll  now  be  able  to  have  a  chance  to  play  the  game  Where  in  Time  is  Carmen 
San  Diego.  Good  luck. 
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Appendix  D  -  Data  Collection  Instrument 
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Abstract 

Girli  and  boys  may  like  different  kindi  of  computer  graphics.  A  gender 
gap  favoring  boys  has  been  reported  in  computer  usage,  Graphics  may 
play  a  role  in  bridging  that  gap  because  of  their  motivational  value.  We 
give  an  overview  of  gender  differences  reported  in  the  literature  regarding 
graphic  content,  color,  detail,  size,  and  movement.  Based  on  our  findings, 
we  propose  preliminary  design  guidelines  for  graphics  in  computer-based 
instructional  materials  to  increase  attention  and  invoke  interest  of  both 
genders.  Cross-cultural  and  age  differences  need  to  be  examined  and  the 
validity  of  the  preliminary  guidelines  tested  to  develop  operational 
guidelines. 
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In  recent  years,  authors  (e.g.,  Krendl,  Broihier,  &  Fleetwood,  1989;  Sutton, 
1991)  have  documented  evidence  of  a  gender  gap  in  computer  usage.  Krendl  ot  al. 
(1989)  reported  that  various  explanations  were  offered,  but  most  authors  agreed 
the  consequences  of  this  gap  would  increase  occupational  and  economic 
inequalities  between  men  and  women .  In  a  three  year  longitudinal  study  they 
found  a  strong  tendency  among  boys  to  respond  favorably  to  computer-based 
instruction  and  pursue  out-of-school  computer  activities.  In  contrast,  the  girls 
tended  to  appear  intimidated  by  the  technology  and  not  pursue  out-of-school 
computer  activities. 

Illustrations  play  a  nugor  role  in  motivating  readers  to  pick  up,  browse  through, 
and  read  books  (Duchastel,  1978).  Dwyer  (1988)  reported  that  visualisation  in  the 
instructional  process  increased  learner  interest,  motivation,  curiosity,  and 
concentration.  Graphics  then,  may  play  a  similar  role  in  computer-based 
instruction.  Just  a  little  over  a  decade  ago  graphics  in  instructional  computer 
software  were  restricted  by  hardware  to  low  resolution  (block),  monochrome 
images.  Since  then  rapid  advances  in  the  technological  capabilities  of  desktop 
computers  have  hurled  software  developers  through  numerous  advances  in 
screen  resolution,  color  palettes,  image  processing  speed,  available  memory,  and 
production  tools.  These  advances  have  resulted  in  instructional  software  that 
often  has  a  heavy  visual  element.  However,  little  notice  has  been  given  to  critical 
issues  in  the  design  of  computer  graphics  such  as  gender  differences  in 
children's  creation  and  preferences  for  graphics  and  illustrations. 

The  following  review  of  the  literature  suggests  that  boys  and  girls  differ  in 
their  creation  and  preference  of  certain  graphic  elements.  Freedman  (1989) 
observed  that  changes  in  fifth  grade  girls'  images  were  often  subtle,  they  seemed 
more  concerned  than  boys  with  the  use  of  color,  whether  colors  went  together, 
and  often  discussed  relational  size  and  arrangement  of  shapes  .  The  boys  were 
more  concerned  with  movement  and  sometimes  conceptualized  shapes  as  objects 
capable  of  violence. 

Jones  (1987)  found  that  preschool  girls'  designs  contained  greater  variety  and 
detail  than  those  of  boys,  and  that  girls  significantly  verbalized  more  than  boys 
when  referring  to  color  selections. 

In  another  study  with  preschoolers  (Calvert,  1989),  where  computer  objects 
such  as  animals  and  vehicles  were  randomly  assigned  sounds  and  actions 
(animation),  boys  chose  more  often  action  objects  than  non-action  objects  while 
the  girls  did  not 

The  computer  program  Storybook  Wanvftr;  WorM-nf-Mfllte-Balief  (MECd 
1991),  allows  users  to  create  stories  and  illustrate  them  with  professionally 
prepared  art  images.  The  designer  of  that  program,  (J.  Sharp,  personal 
communication,  March  10, 1992),  had  observed  gender  differences  in  stories  and 
graphics  children  created  with  the  program.  She  described  girls'  art  as  being 
"friendlier"  than  boys  art,  using  for  example  animals  and  nature  images, 
brighter  colors,  and  daytime  scenes.  The  boys'  art  was  "scarier,"  night  scenes 
were  popular,  often  with  pairs  of  eyes  lurking  in  the  background.  Goblins,  and 
sea  mammals  were  prominent  among  the  creatures  boys  chose  to  use  in  their 
pictures,  and  clouds,  storm,  lightning,  and  fire  among  other  art  pieces  more 
characteristic  of  boys'  graphic  creations  than  girls'. 

Other  studies  and  observations,  although  not  examining  computer  graphics, 
support  these  findings,  revealing  considerable  gender  differences  regarding 
content,  color,  detail,  size,  and  movement  of  objects.  A  description  of  them 
follows. 
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Striking  differences  have  been  found  in  the  content  of  graphics  created  by 
elementary  school  children  (Feinburg,  1977;  McNiff,  1982).  In  McNifFs  analysis 
of  over  1800  drawings  by  26  children,  the  girls'  art  was  described  as  quiet  and 
socially-oriented  with  more  people,  plants,  and  animals  with  the  exception  of  sea 
animals.  The  boys'  art  was  more  "active"  than  girls'  art  and  object-oriented  with 
machines,  such  as  cars  and  motorcycle  in  chases  and  races,  and  conflicts,  for 
example  struggles  and  battles. 

Regarding  the  subjects  of  conflict  and  help,  Feinburg  (1977)  found  large 
differences.  Conflict  for  the  girls  was  seen  as  interpersonal  conflict,  usually  with 
two  friends  or  relatives  in  direct  confrontation.  On  the  other  hand,  boys  drew 
structured  situations,  involving  teams,  and  armies.  Portraying  helping, 
personal  assistance  was  dominant  for  both  girls  and  boys.  However,  girls' 
responses  were  more  closely  related  to  family  and  friends  while  boys  often 
included  rescue  and  danger. 

Gender  differences  regarding  content  are  not  only  reflected  in  graphics  but 
also  in  written  materials.  Trepanier-Street  and  Romatowski  (1991)  analyzed  for 
example  stories  by  180  children  in  grades  1  to  4.  They  found  that  characters  in 
stories  by  girls  showed  more  often  pro-social  behavior  than  characters  in  boys' 
stories.  The  opposite  was  true  regarding  aggressive  behavior.  Also,  emotional 
statements  were  more  common  in  the  girls'  stories  than  the  boys  stories  while 
high  risk/adventure  themes  were  more  common  in  boys'  stories  as  well  as 
assignment  of  high  intensity  actions  to  the  characters,  and  a  higher  percentage  of 
male  characters  than  the  girls  had  in  theirs.  Similarly,  Libby  ft  Aries  (1989) 
analyzed  fantasy  narratives  by  42  preschoolers.  They  found  that  girls  introduced 
significantly  more  female  characters  and  friendly  figures  who  offered  assistance 
while  boys  introduced  significantly  more  male  characters,  aggressive  behavior 
and  attempts  to  master  situations  through  the  use  of  aggressive  activity.  In 
another  recent  study  by  Haynes  and  Richgels  (1992),  children  in  grade  four 
indicated  their  preference  for  fictitious  annotated  book  titles.  Girls'  highest 
ranked  factors  included  fantasy  items  and  items  about  growing  up  and  among 
their  other  top  choices  were  two  historical  nonficttons,  a  social  studies  item  about 
people  in  the  U.S.,  and  a  science  item  about  weather,  plants,  experiments,  and 
animals.  Boys'  highest  ranked  factors  included  items  about  adventure,  space, 
science  and  sports  and  among  other  top  choices  were  items  about  mystery, 
suspense,  and  realistic  animal  fiction.  Items  that  appeared  high  on  the  list  for 
both  genders  were  traditional  and  modern  fantasy,  realistic  fiction,  romance,  and 
adjustment,  historical  fiction,  scientific  items  and  biographies. 

Findings  regarding  content  imply  that  peaceful  looking  graphics  with  people, 
plants,  and  animals  tend  to  be  of  high  appeal  to  girls,  while  graphics  with  figures 
in  actions,  vehicles,  and  implied  conflict,  danger  or  suspense  tend  to  be  of  high 
appeal  to  boys. 

Color 

A  color  preference  study  (Child,  Hansen,  ft  Hornbeck,  1968),  with  more  than 
1,100  students  in  grades  1  to  12,  found  that  girls  consistently  preferred  lighter 
colors  than  the  boys  did  when  looking  at  35  pairs  of  colored  squares.  An  exception 
was  the  color  red.  However,  Guilford  and  Smith  (1959)  did  not  find  a  difference  in 
preference  for  lighter  colors.  They  tested  over  200  colors  in  their  study  with  40 
young  adults.  However,  women  in  the  study  gave  overall  lower  ratings  for  colors 
than  men  did. 
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Results  from  studies  regarding  color  preferences  are  not  very  generalizable. 
The  colors  used  vary  from  study  to  study  and  the  total  number  of  possible  colors  is 
huge  with  differences  in  the  three  perceptual  dimensions  of  color:  hue;  lightness 
or  value;  and  saturation.  Also,  numerous  other  factors  affect  people's 
preferences,  such  as  the  conditions  in  the  study,  current  fashion,  cultural 
differences,  difference  in  color  perception,  and  the  type  of  colored  materials  used. 
A  blue  color  that  most  people  may  find  nice  on  a  small  "objective''  square  may  not 
be  the  first  preference  for  a  whole  room,  or  look  appetizing  in  a  pasta  dish. 

Other  studies  involving  color,  have  found  that  females  have  more  words  to 
describe  colors  than  males  (Lynch  &  Strauss-Noll,  1987),  and  female-interest 
pictures  elicited  significantly  more  color  reference  from  both  women  and  men 
than  male-interest  and  neutral-interest  pictures  (Crawford  &  Chaffin,  1987). 

Based  on  these  findings  it  seems  that  girls  may  be  more  interested  in  colors 
than  boys;  but  possibly  more  critical  of  colors,  and  whether  they  go  together.  Girls 
may  also  prefer  lighter  colors  than  boys  do. 

nfttAii  and  Ska  of  QMaeta 

Goodenough  (1926)  developed  a  popular  method  to  measure  children's 
intellectual  maturity.  She  analysed  and  compared  drawings  of  a  man  by  several 
thousand  children  of  different  ages.  She  found  that  girls  drew  more  items  and 
with  more  detail  than  the  boys. 

Another  finding  showed  that  female  college  and  high  school  students 
remembered  more  detail  on  photographs  in  a  recall  condition  than  males, 
especially  relating  to  persons  (McGuinness  k  McLaughlin,  1982). 

It  seems  obvious  that  details  in  drawings  must  be  tied  closely  to  content,  with 
increasing  detail  appearing  for  subject  matters  the  drawer  knows  well  or  has  a 
high  interest  for.  For  example,  a  study  done  with  6  to  11  year  olds  found  that  boys' 
recall-maps  of  their  home  area  were  richer  in  detail  and  more  accurate  than 
those  of  girls  of  a  similar  age  (Matthews,  1986). 

Size  is  also  a  variable  that  may  be  closely  related  to  content  One  study  found 
that  girls  drew  people  bigger  than  do  boys  (Falchikov,  1990).  This  may  reflect 
social-orientation  and  the  importance  girls  seem  to  place  on  people  in  their 
drawings. 

Based  on  these  findings,  detail  and  possibly  size  of  graphic  objects  of  high 
interest  may  be  appreciated  by  both  genders.  At  least,  this  seems  to  apply  to  detail 
and  size  of  people  and  people-related  items  of  high  interest  to  females. 

Movement  and  Dimension 

Consistent  gender-differences  favoring  males  have  been  found  in 
performance  on  spatial  tasks  in  many  studies,  summarized  for  example  by 
Harris  (1981). 

Goodenough  (1926)  found  that  two  characteristics  of  boys'  drawings  were 
better  sense  of  proportions  than  girls,  and  the  inclusion  of  figures  represented  in 
profile  as  walking  or  running.  The  drawings  analyzed  by  McNiff  (1982)  also 
showed  there  was  more  "action"  in  boys'  art. 

Judging  from  these  findings  and  observations  and  results  reported  earlier 
(Freedman,  1989;  Calvert,  1989),  we  expect  items  that  can  be  moved,  are 
animated,  or  at  least  have  implied  action  associated  with  them  on  a  still  image,  to 
be  cf  high  appeal  to  boys.  Three-dimensional  looking  graphics  may  also  be  more 
appreciated  by  boys  than  girls. 
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Some  Preliminary  OniHftlinft*  for  Onmputor  flrnphic  Dft«yn 

The  finding!  from  our  review  indicate  that  creating  graphics  that 
appeal  to  girli,  designer!  ihould  coniider: 

1.  Including  picturea  of  people,  planti,  and  animali.  (Bated  on: 
Haynei  &  Richgeli,  1992;  McNiff;  1982.) 

2.  Making  picture!  of  people,  plant!  and  animal!  relatively  large  and 
detailed.  (Baied  on:  Falchikov,  1990;  Goodenough,  1926;  Jones,  1987; 
McGuinnei!  &  McLaughlin,  1982.) 

3.  Including  a  variety  of  colon,  not  just  primary  ones,  possibly  in 
lighter  shades,  or  providing  opportunity  to  manipulate  color.  (Based 
on:  Child,  Hansen,  &  Hornbeck,  1968;  Freedman,  1989;  Jones,  1987; 
Lynch  &  Strauss-Noll,  1987.) 

4.  Including  female  characters.  (Based  on:  Libby  &  Aries,  1989; 
Trepanier-Street  &  Romatowski,  1991) 

5.  Including  peaceful,  "friendly"  scenes.  (Based  on:  Libby  k  Aries, 
1989;  McNiff,  1982;  Sharp,  personal  communication,  1992). 

Creating  graphics  that  appeal  to  boys  designers  should  consider: 

1.  Showing  figures  in  action.  (Based  on:  Feinburg,  1977;  Goodenough, 
1926;  McNiff,  1982.) 

2.  Providing  opportunity  to  move  (or  animate)  objects,  or  implying 
movement  of  objects  (Based  on:  Calvert,  et  al.,  1989;  Freedman, 
1989.) 

3.  Including  vehicles.  (Based  on:  McNiff,  1982.) 

4.  Including  male  characters.  (Based  on:  Libby  &  Aries,  1989; 
Trepanier-Street  &  Romatowski,  1991.) 

5.  Including  elements  of  implied  suspense/danger.   (Based  on: 
Feinburg,  1977;  Freedman,  1989;  Haynes  &  Richgels,  1992;  McNiff, 
1982;  Sharp,  personal  communication,  1992;  Trepanier-Street  & 
Romatowski,  1991.) 

More  research  is  needed  before  operational  guidelines  can  be  developed. 
Cross-cultural  and  age  differences  need  to  be  examined.  The  validity  of  the 
preliminary  guidelines  has  to  be  tested.  Many  questions  are  unanswered.  For 
example,  should  a  graphic  be  developed  with  mixed  elements  of  appeal  for  both 
genders,  or  would  that  result  in  graphics  that  no  one  liked?  Should,  instead, 
separate  graphic  treatments  in  the  same  instructional  material  be  offered  or 
different  types  of  graphics  within  the  same  treatment?  Can  catering  to  aesthetic 
preferences  in  some  cases  have  a  negative  effect  on  learning?  However,  in  the 
past,  girls  may  have  been  much  loss  considered  as  an  audience  for  computer 
software  than  boys.  Using  design  guidelines  like  the  ones  we  propose,  for 
graphics  in  educational  software,  may  help  invoke  attention  and  interest  of 
students  of  both  genders. 
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HINTS  AND  LEARNER  CONTROL 
FOR  METACOGMITXVE  STRATEGIES  IN  PROBLEM  SOLVINO 


This  paper  focuses  on  two  types  of  instructional 
design  strategies  (i.e.,  hints  and  learner  control) 
that  can  be  used  to  enhance  metacognitive  strategies  in 
problem  solving.  Hints  are  often  referred  to  as  cues, 
clues,  prompts  or  facilitators,  and  learner  control  as 
student  control  or  internal  locus  of  instructional 
control.  In  this  paper,  the  terms,  hints  and  learner 
control  are  used. 

Metacognition  refers  to  the  active  monitoring  and 
consequent  self -regulation  for  which  one's  own 
cognitive  mental  activities  become  the  objects  of 
reflection  (Plavell,  1987).  According  to  Meichenbaum, 
Burland,  Gruson  and  Camerson  (1985),  "metacognition 
refers  to  cognitions  about  cognitions  or  the  executive 
decision-making  process  in  which  the  individual  must 
both  carry  out  cognitive  operations  and  oversee  his/her 
progress"  (p.5).  It  has  a  means-ends  character,  that 
aims  to  reflect  upon,  evaluate  and  guide  cognitive 
activities  in  an  effective  manner  (Perkins,  Simmonson 
and  Tishman,  1989). 

Representative  activities  of  metacognition  include: 
analyzing  and  characterizing  a  problem;  understanding 
what  one  needs  to  know;  reflecting  upon  what  one  knows 
or  does  not  know  in  order  to  get  to  the  solution; 
making  a  plan  for  the  solution;  using  one's  cognitive 
resources  and  time  on  the  basis  of  a  plan  made  ahead; 
monitoring  progress;  evaluating  the  outcomes  of  the 
plan;  and  revising  the  plan  (Brown,  1987;  Gagne  and 
Glaser,  1987).  In  learner-controlled  instruction, 
similar  learning  activities  are  involved,  by  requiring 
learners  to  keep  on  a  path  of  active  self -regulation, 
and  to  purposefully  use  control  options.  Additionally, 
hints  can  speed  and  facilitate  such  processes  of 
learning.  Metacognitive  strategies  acquired  through  the 
instruction  supported  by  hints  and  learner  control  are 
those  that  are  useful  for  problem-solving. 

In  the  following  portion  of  this  paper,  detailed 
discussion  on  hints  and  learner  control  will  be 
provided • 


HINTS 

what  they  are 

Those  who  attempt  to  solve  problems  need  some 
guidance,  which  can  lead  them  to  the  correct  solution 
throughout  the  whole  process.  Such  guidance  can  be 
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provided  in  a  form  of  hints.  According  to  Sabban 
(1985),  hints  can  serve  the  following  roles:  1)  hints 
can  stop  the  ongoing  action  or  thought;  2)  hints  can 
provide  a  stimulant  for  the  correct  solution;  3)  hints 
can  intentionally  mislead  solution  processes;  and  4) 
hints  can  serve  as  a  measure  of  success  in  problem 
solving. 


why  they  are  important 

Instruction  for  higher-order  problem-solving  skills 
requires  continuous  trial  and  error,  and  the  higher- 
order  skills  are  compatible  to  learning  strategy, 
metastrategy  or  metacognitive  strategy  (Derry  and 
Murphy,  1986).  For  such  instruction,  hints  can  provide 
the  best  opportunity  to  initiate  a  new  direction 
towards  a  correct  solution  and  thus  to  facilitate 
learning  when  a  problem  solver  faces  trouble  in 
progress  (Sabban,  1985).  t 

Hints  can  provide  not  only  information  but  also 
stimulus  for  accurate  problem  solving  by  helping 
learners  identify  the  major  factors  and  underlying 
intention  of  a  problem,  reorganize  the  problem  for  the 
solution,  retrieve  related  skills  or  knowledge,  select 
appropriate  strategies,  and  conduct  problem- solving. 
The  intent  of  hints  is  to  encourage  learners  to  work 
within  an  externally  supported  learning  environment,  in 
which  they  can  conduct  knowledge  compilation,  overcome 
the  problem  of  inert  knowledge  and  production 
defiaienay,  and  develop  metacognitive  inner-dialogue 
strategies. 

Hints  can  activate  knowledge  compilation  processes, 
the  translation  processes  of  declarative  knowledge  to 
procedural  knowledge.  According  to  Gagne  (1985),  one  of 
the  two  major  types  of  procedural  knowledge  is  action- 
sequence,  which  indicates  the  procedures  to  carry  out 
sequences  of  symbolic  operations.  In  developing  action- 
sequence  procedures,  knowledge  compilation  processes 
are  required.  Gagne  argues  that  external  prompts  can 
support  the  compilation  processes  by  guiding  sequential 

steps.  ... 

Also,  hints  can  provide  learners  with  cognitive 
support,  that  can  compensate  for  the  deficiency  in 
their  cognitive  processing.  While  difficulties  that 
learners  face  during  problem-solving  are  often  due  to 
missing  knowledge,  in  many  cases  those  are  due  to  inert 
knowledge  or  fragile  knowledge,  failing  to  retrieve 
related  knowledge  or  to  correctly  combine  retrieved 
knowledge  (Perkins,  Schwartz  and  Simmons,  1988). 
Especially,  developmentally  immature  learners  often 
exhibit  production  deficiencies,  the  failure  to  access 
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and  employ  proc^^r^ng  capabilities  that  they  may 
actually  possess  (Brown,  1974).  Production  deficiencies 
may  occur  due  to  the  learners'  failure  to  recognize  the 
usefulness  of  a  particular  set  of  skills  in  a  certain 
learning  situation  as  well  as  the  desire  to  perform 
such  skills. 

While  missing  knowledge  needs  more  review  and 
practice  to  consolidate  the  knowledge  base,  inert 
knowledge  (and  production  deficiency  as  well)  requires 
special  instructional  remedy  so  that  learners  can 
employ  knowledge  and  skills  to  work  with  (Perkins, 
Schwartz  and  Simmons,  1988).  According  to  Perkins, 
Schwartz  and  Simmons,  hints  can  be  integrated  into  the 
design  of  instruction  for  the  special  remedy. 

In  addition,  hints  can  encourage  learners  not  only 
to  recall  and  apply  previously  acquired  learning  skills 
but  also  to  make  such  process  automatic.  According  to 
Derry  and  Murphy  (1986),  the  enhancement  of  learning 
abilities  requires  the  development  of  not  only  specific 
learning  skills  but  also  executive  control  skills  that 
automatically  accesses  and  combines  skills  needed  for 
learning.  This  can  be  applied  to  problem-solving 
abilities  as  well.  On  this  point,  hints  can  help  the 
enhancement  of  executive  control  skills  (Bonk  and 
Reynolds,  1992).  Further,  according  to  Derry  and  Murphy 
(1986),  executive  control  skills  are  those  that 
maintain  and  transfer  metacognitive  strategies. 


how  to  provide  them 

A  theme  emerged  repeatedly  in  research  conducted 
from  a  metacognitive  perspective  is  that  executive 
control  skills  cannot  be  trained  directly,  but  must  be 
gradually  developed  and  automated  over  a  long  period 
(Derry  and  Murphy,  1986).  Hints  can  be  used  in 
designing  guided  instruction  for  the  training  of 
executive  control  skills. 

Many  of  metacognitive  strategies  related  to 
executive  control  skills  cannot  be  made  explicit. 
Moreover,  even  given  explicit  strategies,  the  use  of 
the  strategies  highly  depends  on  the  way  in  which  they 
are  embedded  in  instruction  (Collins,  Brown  and  Newman, 
1986). 

Because  of  these  problems,  Collins,  Brown  and  Newman 
argue  that  instruction  should  be  designed  to  provide 
learners  with  the  opportunity  to  observe  and  engage  in 
using  an  expert's  strategies  and  related  knowledge  to 
help  them  understand  how  these  strategies  and  knowledge 
fit  together  and  cue  off.  On  this  point,  they  suggest 
the  cognitive  apprenticeship  method  which  reflects 
modeling-coaching- fading  paradigm.  This  paradigm  is 
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designed  to  help  learners  acquire  an  integrated  set  of 
metacognitive  as  well  as  cognitive  strategies  through 
observation  and  guided  practice. 

Modeling  involves  the  demonstration  of  an  expert's 
task  performance,  which  can  help  learners  observe  and 
build  a  conceptual  model  of  the  task  performance. 
Coaching  consists  of  observing  learners  while  they 
carry  out  a  task  and  offering  help  as  needed.  Fading 
refers  to  the  gradual  removal  of  the  help  as  learners 
can  perform  the  task  on  their  own. 

In  integrating  hints  into  the  design  of  problem- 
solving  instruction,  an  expert's  uses  of  hints  can  be 
demonstrated  by  prescribed  instruction  or  by  a  teacher. 
Then,  learners  can  have  the  opportunity  to  use  the 
hints  as  they  carry  out  a  given  task,  while  they  can 
get  help  as  needed.  The  processes  of  learners' 
performance  need  to  be  analyzed  to  determine  when  they 
no  longer  need  hints.  When  there  is  an  evidence  of 
automatic  initiation  and  use  of  metacognitive 
strategies,  hints  can  be  gradually  phases  out  from  the 
instruction.  Through  this  learning  process  supported  by 
hints,  learners  can  gain  control  over  their  uses  of 
metacognitive  strategies  in  problem  solving,  and  this 
control  can  become  internalized. 


LEARNER  CONTROL 

what  it  is 

Concerning  the  loci  of  instructional  control,  the 
control  can  be  assigned  either  to  a  program  or  to  a 
learner.  While  program  control  presents  instructional 
components  on  the  basis  of  prescribed  rules  with  little 
choice  given  to  learners,  learner  control  provides 
learners  with  opportunities  to  analyze  their  own 
comprehension  and  needs  and  to  use  instructional 
components  according  to  the  analyzed  needs. 


why  it  is  important 

Advocating  learner  control,  Merrill  (1984) 
conjectures  an  internal  model  of  metacognition  inside 
each  learner  that  directs  and  orchestrates  how  s/he 
uses  control  over  instructional  components  to  enhance 
the  present  state  of  knowledge.  On  the  basis  of  the 
model,  a  learner  makes  decisions  and  examines  how  the 
decisions  work  out,  and  through  these  processes,  s/he 
can  learn  "how  to  learn". 

Under  learner  control,  learners  need  to  be  aware  of 
control  options  and  deliberately  make  decisions  about 
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the  use  of  options.  Therefore,  individual  learners  are 
situated  in  the  position  to  actively  analyze  the  nature 
of  a  task,  to  estimate  their  understanding  of  the  given 
task,  to  select  activities  to  be  performed  in 
accomplishing  the  task,  to  decide  how  to  perform  the 
selected  activities,  and  to  evaluate  the  results  of 
their  decisions,  while  continuously  monitoring  their 
own  cognitive  processes  and  performances  (Holmes, 
Robinson,  and  Steward,  1985;  Merrill,  1984). 

Through  the  instruction,  learners  can  continuously 
practice  conscious  reflection  on  their  cognitive 
abilities,  task  demands  and  learning  strategies,  and 
learn  how  to  manage  their  own  thinking  processes  and 
learning  activities  in  order  to  achieve  desired  goals 
of  instruction.  Thus,  through  the  exploration  of  given 
tasks  and  use  of  self -regulatory  mechanism,  they  can  be 
led  to  enhance  their  abilities  to  perform  and  behave 
strategically. 

As  a  result,  through  learner  controlled  instruction, 
learners  can  be  able  to  integrate  new  learning 
materials  thoroughly  within  their  existing  knowledge 
schema,  and  enhance  their  metacognitive  strategies  in 
an  effective  way  (Merrill,   1984;  Rubincam  and  Olivier, 
1985) . 


hour  to  provide  it 

Significant  efforts  have  been  devoted  to  research  on 
learner  control,  but  the  results  have  been  diverse, 
making  generalization  difficult.  Thus,  although  the 
question  as  to  whether  instructional  designers  should 
assign  control  to  a  learner  or  to  a  program  has  been 
frequently  asked,  it  has  not  been  resolved. 

This  may  be  due  to  the  lack  of  instructional  design 
supports  that  take  off  some  of  the  cognitive  burden 
provided  by  the  unique  learning  environment  and  guide 
learners  to  effectively  go  through  the  instruction  (Jo, 
1991).  On  this  point,  recommendations  made  by  theory 
and  research  need  to  be  integrated  into  the  design  of 
instruction  to  help  learners  consciously  make  decisions 
over  their  own  learning  and  build  effective  learning 
strategies. 

First,  control  options  should  be  clearly  labeled  to 
help  learners  use  control  options  effectively 
(Reigeluth  and  Stein,   1983).  Second,  immediate 
feedback,  continuous  advice  on  learners'  on-going 
progress  and  summaries  of  their  uses  of  control  options 
should  be  presented  to  help  learners  make  "informed 
decisions"  about  their  own  learning  (Hannafin,  1984; 
Kinzie,  1988;  Lepper  and  Malone,   1987).  Third,  basic 
requirement  over  important  instructional  components 
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should  be  provided  to  learners  in  order  to  assure  that 
they  do  not  bypass  the  components  (Kinzie,  1988)  . 
Fourth,  prior  to  instruction,  pretraining  should  be 
provided  to  learners  to  help  them  become  familiar  with 
the  novel  learning  system  with  control  options,  perform 
conscious  cognitive  information  processing,  and 
understand  objectives,  procedures  and  values  involved 
in  building  their  own  learning  strategies  (Hannafin, 
1984;  Holmes  et  al.,  1985;  Kinzie,  1988;  Merrill,  1984; 
Schmitt  and  Newby,  1985).  Schmitt  and  Newby  (1985) 
argue  that  if  learners  learn  when  and  why  they  need  to 
use  such  Strategic  approaches  (conditional  knowledge) 
along  with  learning  what  they  are  (declarative 
knowledge)  and  how  to  use  them  (procedural  knowledge), 
the  learners  can  quickly  gain  competence  and  confidence 
in  their  own  abilities  to  use  the  approaches 
appropriately  and  to  achieve  excellent  outcomes.  Thus, 
pretraining  can  contribute  to  helping  learners  gain 
such  knowledge. 

The  metacognitive  strategies  acquired  through  the 
instruction  supported  by  hints  and  learner  control  may 
be  those  that  are  transferred  to  other  contexts  and 
facilitate  further  learning  activities.  As  learners' 
confidence  increases  through  such  instruction,  they  may 
attempt  to  modify  and  improve  their  own  metacognitive 
strategies  even  when  they  are  not  required  (Kinzie, 
1983) . 

This  paper  has  dealt  with  theoretical  aspects  of  the 
effects  of  hints  and  learner  control  in  enhancing 
: metacognitive  strategies.  Concerning  the  potential  of 
the  two  design  strategies,  much  effort  needs  to  be 
devoted  to  research  in  order  to  verify  the  theory  and 
to  uncover  the  details  of  learning  experiences  gained 
through  the  instruction.  In  addition,  due  to  the 
importance  of  strategic  approaches  guided  by  hints  and 
learner  control,  research  should  attempt  to  monitor 
learners'  conscious  cognitive  activities  involved  in 
their  learning  processes. 
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Abstract 

This  study  compared  the  effects  of  instructor-provided  vs.  learner-generated 
analyses  of  semantic  relationships  between  major  concepts  on  structural 
knowledge  acquisition  in  an  introductory  psychology  course.  One  group  explained 
the  relationships  on  instructor-provided  graphic  organizers;  the  other  group 
classified  relationships  between  concepts  by  completing  skeletal  maps  provided  by 
the  instructor.  Increase  in  structural  knowledge  was  related  to  practice  effect  for 
both  groups;  the  group  that  generated  analyses  scored  higher  at  a  near-significant 
lovol.  Regardless  of  group,  there  was  a  significant  relationship  between  total  exam 
score  and  frequency  of  completing  the  treatment  exercises  (an  indicator  of 
motivation). 


Introduction 

Learning  and  Semantic  Networks 

Our  semantic  networks  represent  our  knowledge  structures  which  enable 
learners  to  combine  ideas,  infer,  extrapolate  or  otherwiso  reason  from  them. 
Learning  consists  of  building  new  structures  by  constructing  new  nodes  and 
interrelating  them  with  existing  nodes  and  with  each  other.  The  more  links  that 
the  learner  can  form  between  existing  knowledge  and  new  knowledge,  the  bettor 
the  learner  will  comprehend  the  information  and  the  easier  learning  will  be. 
Learning,  according  to  semantic  network  theory,  is  the  reorganization  of  the 
learner's  knowledge  structure.  During  the  process  of  learning,  the  learner's 
knowledge  structure  begins  to  resemble  the  knowledge  structures  of  the  instructor, 
and  the  degree  of  similarity  is  a  good  predictor  of  classroom  examination 
performance  (Diekhoff,  1983;  Shavelson,  1974;  Thro,  1978).  Instruction,  then,  may 
be  conceived  of  as  the  learner's  mapping  of  subject  matter  knowledge  (usually  that 
possessed  by  the  teacher  or  expert)  onto  the  learner's  knowledge  structure. 

If  we  accept  the  conception  that  learning  involves  the  reorganization  of  the 
learner's  cognitive  structure,  then  instruction  involves  the  learner's  assimilation 
of  the  expert's  knowledge  structure.  In  order  to  help  students  reorganize  and  tune 
their  knowledge  structures,  we  need  instructional  strategies  for  depicting  and 
displaying  appropriate  knowledge  structures  to  students  and  tools  for  helping  them 
organize  their  knowledge  structures.  Instructional  strategies  may  illustrate  or 
convey  appropriate  knowledge  structures  to  students,  whereas  the  tools  or  learning 
strategies  may  help  learners  to  acquire  and  refine  their  own  knowledge  structures 
(Jonassen,  Beissner,  &  Yacci,  1993). 

Instructional  Strategies  vs.  Learning  Strategies 

Instructional  strategies  and  learning  strategies  lie  on  a  continuum  that 
describes  the  level  of  learner  involvement.  An  instructional  strategy  is  a  method 
or  technique  for  providing  instruction  to  learners.  These  strategies  are 
implemented  through  instructional  tactics  (Jonassen,  Grabinger,  &  Harris,  1991). 
For  instance,  an  instructional  strategy  may  recommend  motivating  the  learner 
prior  to  instruction,  which  may  call  for  tactics  such  as  arousing  learner 
uncertainty,  asking  a  question  or  presenting  a  picture.  A  strategy  aimed  at 
teaching  a  concrete  concept  may  call  for  the  use  of  tactics  such  as  matched  examplo- 
nonexample  pairs  or  deriving  the  criterion  attributes  from  a  set  of  examples. 
Instructional  strategies  provide  the  overall  plan  that  guides  the  selection  of 
instructional  tactics  that  facilitate  learning.  Essentially,  instructional  strategies 
are  mstructor-provided  interventions  that  are  meant  to  constrain  learner 
processing  of  information.    The  instructional  strategy  used  in  this  study 
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graphically  illustrated  key  concepts  and  their  interrelationships  between  thoBe 
concepts,  which  attempted  to  map  the  teacher's  knowledge  structure  onto  the 
learners'  structures. 

Learning  strategies  axe  mental  operations  that  the  learner  may  use  to  acquire, 
retain  and  retrieve  different  kinds  of  knowledge  or  performance.  The 
fundamental  difference  between  instructional  strategies  and  learning  strategies 
is  that  the  former  are  largely  mathemagenic  and  the  latter  are  generative 
(Jonassen,  1985).  Mathemagenic  instructional  strategies  control  the  processing  of 
learners  while  leading  them  to  learning,  so  they  result  in  anticipated  learning 
outcomes.  They  facilitate  acquisition  of  specific  content  knowledge  (intentional 
learning)  but  generally  do  not  facilitate  or  even  impede  the  acquisition  of  any  other 
knowledge  (incidental  learning). 

Learning  strategies,  on  the  other  hand,  are  generative;  that  is,  they  enable 
learners  to  take  an  active,  constructive  role  in  generating  meaning  for 
information  by  accessing  and  applying  prior  knowledge  to  new  material. 
Wittrock's  (1978)  generative  hypothesis  asserts  that  meaning  for  material 
presented  by  computer  or  any  other  medium  is  generated  by  the  learner's 
activating  and  altering  existing  knowledge  structures  in  order  to  interpret  what  is 
presented.  Learning  BtrategieB  are  intended  to  increase  the  number  of  links 
between  presented  information  and  existing  knowledge.  Learning  strategies, 
unlike  instructional  strategies,  are  more  learner-controlled  as  well  as  learner- 
generated,  because  they  engage  the  learners  and  help  them  to  construct  meaningful 
representations.  Their  success  depends  upon  the  learner  taking  an  active  role  in 
controlling  their  use.  The  strategy  that  was  investigated  in  this  study  facilitated 
the  acquisition  of  structural  knowledge. 

Instructional  and  Learning  Strategies  for  Facilitating  Structural  Knowledge 

The  learning  variable  that  was  investigated  in  this  study  was  structural 
knowledge.  Structural  knowledge  is  the  knowledge  of  how  concepts  within  a 
domain  are  interrelated  (Diekhoff,  1983).  Structural  knowledge  enables  learners 
to  form  the  connections  that  they  need  to  describe  and  use  scripts  or  complex 
schemas.  It  is  a  form  of  conceptual  knowledge  that  mediates  the  translation  of 
declarative  knowledge  into  procedural  knowledge.  There  are  a  number  of 
instructional  strategies  for  conveying  structural  knowledge  representations  and  a 
number  of  learning  strategies  for  facilitating  knowledge  acquisition  (Jonassen, 
Beissner,  &  Yacci,  1993).  Instructional  strategies  that  affect  the  way  that  learners 
encode  these  structures  into  memory  include  explicit  graphic,  mapping  techniques, 
such  as  graphical  organizers,  spider  maps,  semantic  maps,  and  causal  interaction 
maps,  for  conveying  knowledge  structures  to  students.  Teachers  and  designers 
may  use  a  number  of  verbal  instructional  techniques,  such  as  content  structures 
and  elaboration  theory,  to  convey  the  underlying  structures  in  materials  to 
students.  There  are  also  a  number  of  learning  strategies  that  learners  can  use  to 
build  their  own  structural  knowledge  representations,  including  pattern  noting 
(Buzan,  1974;  Fields,  1982),  networking  (Dansereau,  Collins,  McDonald,  Holley, 
Garland,  Diekhoff,  &  Evans,  1979)  and  the  node  acquisition  integration  technique 
(Diekhoff,  Brown,  &  Dansereau,  1982). 

Among  these  strategies  is  a  study  strategy  called  the  Frame  Game  that  was 
developed  to  accompany  an  educational  psychology  textbook  (Clifford,  1981).  The 
Frame  Game  is  a  text  processing  strategy  that  identifies  the  most  important  concepts  in 
a  textbook  chapter  and  then  requires  the  learner  to  identify  the  relationships  between 
the  concepts  by  assigning  them  to  predetermined,  mapped  relationships  (see  Fig.  1  for  a 
completed  frame  and  Fig.  2  for  a  frame  to  be  completed  by  students).  This  analysis 
Btrategy  requires  that  learners  search,  contrast,  validate,  elaborate,  confirm,  and  test 
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information  from  the  chapter.  These  relationship  maps  may  be  used  to  engage 
learners  in  generative  processing  of  textual  information  or  to  depict  the  information. 
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Fig.  1.  Completed  frame  for  chapter  in  psychology  textbook. 

In  this  study,  we  compare  (1)  the  provision  of  structural  information  to  learners  as 
a  review  of  the  information  with  (2)  the  learner  analysis  of  the  relationships 
between  the  concepts  in  the  chapter  (For  a  more  complete  discussion,  see  Cole  & 
Jonassen,  1993.) 
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The  patterns  of  frames  in  Fig.  2  represent  different  relationships  between 
concepts,  such  as  hierarchical,  sequential,  and  associates  relationships.  Students 
have  to  analyze  the  concepts  and  decide  which  combinations  can  fit  into  the 
structures.  Such  an  activity  requires  deep-level  semantic  analysis  of  the  main 
ideas  in  the  textbook  and  lectures. 
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Kltf.  2.  Student  frame  from  psychology  textbook. 
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Purpose of  Study 

The  primary  purpose  of  this  study  was  to  clarify  and  extend  the  findings  of  a 
prior  study  (Jonassen,  Cole,  &  Bamford,  1992)  comparing  the  effects  of  a  structural 
knowledge  instructional  strategy  with  the  effects  of  a  structural  knowledge 
learning  strategy  on  the  acquisition  of  structural  knowledge.  In  that  study 
increases  in  structural  knowledge  acquisition  interacted  with  content  difficulty 
and  the  relative  difficulty  of  the  exercises.  Motivation  also  appeared  to  bo  a  factor 
for  one  Group. 

In  this  study,  we  wanted  not  only  to  compare  the  effects  of  providing  graphical 
organizers  in  the  form  of  completed  frames  with  requiring  students  to  complete 
frameB  as  a  study  strategy  prior  to  examinations,  but  also  to  overcome  several 
limitations  of  the  previous  study.  First,  we  wanted  to  ensure  at  least  minimally 
similar  cognitive  processing  of  the  organizer  and  the  frames  by  students  in  their 
respective  groups.  In  the  earlier  study,  students  who  received  the  organizer  had  not 
been  required  to  process  it  in  any  way.  Moreover,  the  students  who  received  the 
organizer  might  have  engaged  in  rote  memorization  or  some  form  of  verbal 
mediation.  Second,  since  incentives  play  an  important  role  in  student 
performance  (Hicken,  Sullivan,  &  Klein,  1992)  we  wanted  to  provide  adequate 
incentive  for  completing  the  exercises.  Third,  since  it  apparently  takes  months  for 
a  new  learning  strategy  to  displace  a  well-established  old  one  (Duffy  &  Roehler 
1989),  we  wanted  to  have  students  utilize  a  single  strategy  for  an  entire  semester;  in 
the  earlier  study  students  used  each  strategy  for  one  exam  in  a  counter-balanced 
research  design.  Fourth,  we  wanted  to  eliminate  the  confounding  effect  of  the 
counterbalanced  design  since  we  could  not  control  the  difficulty  level  of  each 
chapter.  The  results  of  the  prior  study  suggest  that  it  is  more  difficult  to  assimilate 
structural  knowledge  for  some  chapter?  than  for  others.  For  example,  it  is  easier  to 
relate  learning  and  memory  concepts  to  a  class  of  college  students  than  it  is 
concepts  on  physiology.  Finally,  we  wanted  to  include  enough  items  on  each 
subscaie  of  structural  knowledge  to  be  sensitive  to  learner  performances  on  each 
exam. 

Method 

Subjects 

Thirty-eight  students  from  two  sections  of  a  General  Psychology  (Psy  101) 
course  at  a  large  community  college  in  metropolitan  Denver  completed  the  study 
during  eight-week  summer  term,  1992.  Five  other  students  failed  to  complete  all  of 
the  tests  so  they  were  dropped  from  the  analysis.  There  were  21  subjects  in  Group  1 
and  17  in  Group  2.  p  ' 

Instruments 

Four  subject-matter  exams  were  designed  lor  the  experiment.  Each  exam 
consisted  of  60  questions  worth  one  point  each,  34  of  which  were  multiple-choice 
questions  testing  recall  and  comprehension  of  the  text  and  lecture  material  In 
order  to  assess  structural  knowled*'  acquisition  following  various  treatments,  we 
developed  three  subacales  to  measure  different  aHpccts  of  structural  knowledge-  (a) 
10  relationship  proximity  judgments,  (b)  8  semantic  relationships,  and  (c)  8 
analogies.  All  of  the  structural  knowicdKe  test  questions  were  developed  to  focus  on 
rolationHhips  between  important  concepts  contuined  in  the  textbook  chapters 

The  relntioiiHhip  proximity  jud:  ,onts  required  that  students  judge  the 
strength  of  the  relutionship  between  tv.o  terma  and  assign  a  number  between  1  and  9 
to  each  of  several  pairs  of  concepts  to  indicate  how  strong  a  relationship  they  thought 
existed  between  the  concepts  in  each  pair  (Diekhoff,  1983;  Schvaneveldt,  Durso 
Goldsmith,  Breen,  Cooke,  Tucker  &  DeMaio,  1985;  Shavelson,  1972).  For  example 
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2.  endorphins  —  amygdala 


3.  cerebral  cortex  —  Broca's  area 


  4.  seratonin  —  temporal  lobe 

The  semantic  relationships  subscale  consisted  of  eight  multiple-choice 
questions  that  required  students  to  understand  and  recognize  the  nature  of  the 
relationship  between  two  concepts.  These  relationships  were  paraphrased  from  tho 
textbook  or  lecture.  For  example: 

I 

 16.  sensory  registers  ....  short  term  memory 

a.  precedes 

b.  is  defined  by 

c.  is  independent  of 

d.  is  inferred  by 

  17.  acoustic  ....  memory 

a.  results  in 

b.  is  independent  of 

c.  is  opposite  of 

d.  is  an  example  of 

Finally,  the  analogies  subscale  required  students  to  complete  eight  analogies 
consisting  of  four  of  the  concepts  from  the  textbook  or  lecture.  For  example: 

  22.  decay  :  forgetting  ::  :  retrieval 

a.  mood 

b.  repression 

c.  rehearsal 

d.  primacy 

 23.  acquisition  :  extinction  ::  punishment  :  

a.  negative  reinforcement 

b.  positive  reinforcement 

c.  variable  reinforcement 
d    classical  conditioning 

These  three  types  of  questions  were  used  to  assess  structural  knowledge 
acquisition.  In  order  to  provide  standards  for  assessment,  the  researchers  agreed 
on  the  answers  to  each  of  these  questions.  Answers  to  the  multiple-choice  questions 
were  recorded  on  the  front  of  Scantron  answer  sheets;  semantic-proximity  items 
were  written  on  the  back.  Exams  covered  history  and  methodology  of  psychology 
(chapters  1  &  2),  physiology,  sensation  and  perception  (chapters  3  &  4),  learning 
and  memory  (chapters  6  &  7),  and  motivation,  emotion,  and  states  of 
consciousness — primarily  sleep  (chapters  5,  9,  &  10). 

Materials 

Researcher-generated  study  maps  were  generated  for  each  chapter;  mapping 
allowed  students  to  visualize  the  relationships  among  terms  —  superordinate/ 
subordinate/coordinate  (hierarchical)  and  sequential  organization— as  well  aa  the 
depiction  of  the  terms  in  classes,  analogies,  similarities,  cause  and  effect,  and 
opposites  (see  Fig.  1).  Each  map  alphabetically  listed  all  the  terms  used  in  the 
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frames.  Students  were  also  instructed  to  study  the  list  of  terms  at  the  end  of  each 
chapter  in  the  text. 

Templates  of  maps  were  designed  for  a  student-generated-mapping  exercise  to 
be  assigned  prior  to  Exam  2.  These  maps  (see  Fig.  2)  were  the  same  as  the 
instructor-provided  maps  in  Fig.  1  except  that  most  of  the  frames  in  the  maps  were 
empty,  requiring  the  student  to  fill  in  the  appropriate  concepts.  Since  a  prior  study 
(Jonassen,  Cole,  &  Bamford,  1992)  indicated  that  having  extra  terms  in  the  list 
made  the  cognitive  load  too  great,  the  lists  in  this  study  included  only  the  terms 
actually  used  in  the  frames.  To  further  optimize  the  cognitive  demands  on  the 
students,  the  templates  provided  at  least  one  term  in  each  set  of  frames  (always  one 
in  each  pair  of  terms  in  an  analogy). 

Procedure 

The  study  was  a  quasi-experimental  design  using  intact  psychology  classes. 
The  classes  met  150  minutes  twice  a  week  at  9:50  a.m.  (Group  1)  and  6:30  p.m. 
(Group  2)  for  the  duration  of  an  8-week  summer  term.  One  of  the  researchers 
served  as  instructor  for  both  sections  of  the  course. 

A  repeated-measures  design  was  utilized  to  assess  learning  across  the 
semester  and  to  measure  group  differences. 

Group  1  Group  2 

Exam  1  Maps  Provided  Maps  Provided 

Exam  2  Maps  Provided  Maps  Generated  by  the  Class  as  a  Whole 

Exam  3  Maps  Provided  Maps  Generated  in  Small  Groups 

Exam  4  Maps  Provided  Maps  Generated  Individually 

Prior  to  Exam  1,  students  acquired  mapping  concepts  and  procedures  by 
completing  a  generic  mapping  exercise.  This  exercise  introduced  students  to  the 
mapping  process  that  they  would  encounter  throughout  the  semester.  Students  were 
told  that  they  would  be  involved  in  a  study  about  using  learning  strategies. 

One  week  prior  to  Exam  1,  a  study  sheet  containing  key  terms  from  lecture  and 
text  was  distributed.  The  class  prior  to  the  exam  was  partially  devoted  to 
explaining  the  key  terms'  relationships  within  the  instructor-provided  maps  that 
were  distributed  to  both  groups.  Exam  I  covered  chapters  1  and  2  (history  & 
methodology  of  psychology). 

One  week  prior  to  Exam  2,  a  study  sheet  containing  key  terms  from  lecture  and 
the  text  was  distributed  to  both  groups.  A  portion  of  the  class  period  immediately 
prior  to  the  exam  was  devoted  to  the  mapping  exercise.  Group  1  received  instructor- 
provided  maps;  students  were  required  (as  a  homework  assignment)  to  explain  in 
writing  the  mapped  relationships.  Group  2  students  received  the  templates 
designed  for  student-generated-mapping,  and  the  teacher  facilitated  whole-class 
problem  solving  of  how  to  complete  the  maps  correctly.  Exam  2  covered  chapter  3 
and  4  (physiological  psychology,  and  sensation  and  perception). 

One  week  prior  to  Exam  3,  a  study  sheet  containing  key  terms  from  lecture  and 
the  text  was  distributed  to  both  groups.  As  with  Exam  2,  a  portion  of  the  class  period 
prior  to  the  exam  was  devoted  to  the  mapping  exercise.  As  before,  Group  1  students 
were  required  (as  a  homework  assignment)  to  provide  a  written  explanation  of 
each  graphically  mapped  conceptual  relationship  in  the  instructor-provided  maps 
Group  2  students  received  the  templates  designed  for  student-generated-mapping 
Students  divided  into  groups  of  3-4  students  and  completed  the  mapping  exercise  in 
class.  Exam  3  covered  chapters  6  and  7  (learning  &  memory). 

One  week  prior  to  Exam  4,  a  study  sheet  containing  key  terms  from  the  lectures 
and  the  text  was  distributed  to  both  groups.  As  before,  Group  1  students  were 
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required  to  provide  a  written  explanation  of  each  graphically  mapped  conceptual 
relationship  in  the  instructor-provided  maps.  Group  2  students  received  the 
templates  designed  for  student-generated-mapping.  Group  2  students  were 
required  to  complete  this  exercise  individually  as  a  homework  assignment.  Exam 
4  covered  chapters  5,  9,  and  10  (motivation,  emotion,  and  states  of  consciousness— 
primarily  sleep). 

Following  each  exam,  the  instructor  reviewed  the  exam  in  class  and  provided 
feedback  on  each  student's  test  performance.  Homework  assignments  (written 
descriptions  of  maps  for  Group  1,  and  student-  or  group-generated  maps  for  Group  2) 
were  assessed  for  degree  of  completion;  the  homework  assignments  altogether 
counted  10%  of  the  course  grade. 

At  the  end  of  the  semester,  students  were  administered  a  questionnaire 
designed  to  assess  their  perceptions  of  the  mapping  strategies  that  were  presented  in 
class. 

Exam  items  included  34  general  recall/comprehension  items,  and  26 
structural-knowledge  items— 8  analogies,  8  relationships,  10  proximities— for  a 
total  of  60  questions  per  exam.  Scores  on  each  dependent  measure  of  interest  were 
converted  to  percentages  so  that  they  could  be  compared  across  scales  and 
subscales.  These  dependent  measures  were  all  analyzed  using  a  repeated- 
measures  analysis  of  variance  (ANOVA).  Because  we  assumed  that  students  in 
each  treatment  were  learning  to  apply  their  respective  mapping  strategies  on  the 
exercises  for  Exams  1  and  2,  analyses  focused  on  Exams  3  and  4. 


Results 


Total  Exam  Scores 

An  ANOVA  of  the  total  exam  scores  for  Exams  1  and  2  indicated  no  statisti- 
cally significant  differences  between  Groups  1  and  2.  This  was  the  period  in  which 
the  learners  were  practicing  their  respective  skills;  so  analyses  focused  on  Exams 
3  and  4.  No  significant  differences  in  the  total  scores  occurred  between  groups  or 
between  exams  (see  Fig.  3). 
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Fig  3.  Total  exam  scores 

Group  1  (teacher-provided)  scores  declined  five  percentage  points  on  Exam  4, 
while  the  mapping  group's  score  increased  slightly.  However,  the  interaction  was 
not  significant  (p  =  .10).  Next  we  wanted  to  determine  if  completion  of  the  assigned 
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exercises  contributed  to  total  exam  performance.  The  total  number  of  maps  com- 
pleted by  all  subjects  were  summed  and  classed  into  three  groups:  those  who  com- 
pleted only  a  Few  of  the  maps,  those  who  completed  Most  of  the  maps,  and  those  who 
completed  All  maps.  Total  scores  were  significantly  different  between  these 
groups  (ilf=  67, 78, 85  for  Exam  3  and  68,  72,  and  85  for  Exam  4  for  the  Few,  Most,  and 
All  groups  respectively;  F  =  7.4,  p  <  .01),  but  the  treatment  group  by  completion  group 
was  not  significant.  The  treatments  did  not  appear  to  differentially  affect  total 
exam  performance.  Next  we  analyzed  the  three  subscales  for  structural  knowledge 
(proximities,  semantic  relationships,  and  analogies). 

Structural  Knowledge  Performance 

Proximities.  Student  proximity  scores  were  compared  with  experts'  scores, 
yielding  a  correlation  coefficient  for  the  10  proximity  items  on  each  exam.  These 
coefficients  were  compared  using  ANOVA  (see  Figure  4). 
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Figure  4.  Total  proximities  score 

A  significant  main  effect  occurred  for  treatment  (p  <  .05),  with  the  mapping-group 
scores  (M  =  79)  exceeding  the  teacher-provided  group  scores  (AT  =  69).  The  repeated 
measures  by  group  interaction  neared  significance  (p  =  .06),  with  the  teacher- 
provided  group's  scores  remaining  stable  between  Exams  3  and  4  and  the  mapping 
group's  scores  declining  significantly.  This  difference  becomes  even  more 
dramatic  in  a  comparison  of  intentional  learning  (mapped  items)  and  incidental 
learning  (unmapped  items;  see  Fig.  5). 
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Fig.  5.  Mapped  and  unmapped  proximities 

On  mapped  proximity  items,  the  teacher-provided  group's  scores  increased 
significantly  while  the  mapping  group's  scores  decreased  significantly  (F  =  6.3,  p 
<  .05).  The  interaction  on  the  unmapped  proximities  (incidental  learning)  was  not 
significant.  This  indicates  that  the  content  of  Exam  4  was  more  difficult  than 
Exam  3  and  that  the  mapping  group  was  impeded  by  some  aspect  of  that  Exam. 

Semantic  relationships.  On  the  relationship  items,  there  was  a  significant 
repeated  measures  effect  (F  =  10.4,  p  <  .01;  see  Fig.  6)  and  a  significant  difference 
between  those  who  completed  Few,  Most,  or  All  of  the  mapping  exercises  (p  <  .01  for 
both  exams). 
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Fig.  6.  Total  relationship  scores 

The  scores  of  both  groups  increased  from  Exam  3  to  Exam  4,  indicating  a  signifi- 
cant practice  effect.  As  with  the  proximity  items,  those  who  completed  All  of  the  ex- 
ercises outperformed  those  who  completed  Most  and  those  who  completed  just  a  Few 
of  the  map-related  exercises  assigned  during  the  term.  Unlike  the  proximity 
items,  there  was  a  significant  repeated  measures  by  group  interaction  on  inten- 
tional learning  (mapped  items),  with  the  teacher-provided  group  maintaining  a 
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steady  performance  between  Exams  3  and  4  and  the  mapping  group  increasing  its 
performance  (see  Figure  7).  There  was  a  significant  repeated  measures  effect  for 
incidental  learning,  with  both  groups  improving  their  performances  between 
Exams  3  and  4  (see  Figure  7). 
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Fig.  7.  Mapped  and  unmapped  relationships 

Analogies.  There  was  a  significant  repeated  measures  by  treatment 
interaction  on  the  analogies  (F  =  6.7,  p  <  .05),  with  the  teacher-provided  group 
declining  and  the  mapping  group  improving  (see  Fig.  8).  There  were  too  few 
mapped  analogies  for  any  meaningful  interpretation  of  intentional  learning. 
However,  there  was  a  significant  repeated  measures  by  treatment  interaction  for 
incidental  learning  (F  =  6.7,  p  <  .05),  with  the  teacher-provided  group  declining 
and  the  mapping  group  improving  (see  Fig.  8). 
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Figure  8.  Analogy  Scores 
Non  Structural  Knowledge  Performance 

The  first  34  items  of  the  test  were  predominantly  recall  and  comprehension 
items.  There  was  a  significant  practice  effect  for  the  total  score  on  this  section  of 
the  test  (F  =  8.5,  p  <  .01)  as  well  as  on  the  recall  items  (F  =  27.8,  p  =  .0001).  Both 
groups  declined  on  both  measures  between  Exams  3  and  4  (see  Fig.  9). 
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Fig.  9.  Non-structural  knowledge  total  scores 

A  repeated  measures  ANOVA  revealed  that  both  groups  declined  significantly 
in  intentional  learning  (F  =  4.7,  p  <  .05),  and  incidental  learning  (F  =  14.6,  p  .0005; 
see  Figure  10).  These  findings  confirm  the  above  interpretation  that  the  content  of 
Exam  4  was  more  difficult  than  the  content  of  Exam  3. 
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Fig.  10.  Mapped  and  unmapped  non-structural  knowledge 


Discuasion 

The  most  consistent  result  is  based  upon  the  number  of  mapping  exercises 
completed  by  each  group.  Confirmatory  regression  analyses  regressing  the  num- 
ber of  exercises  completed  onto  each  of  the  dependent  variables  resulted  in 
significant  predictive  value  for  the  number  of  exercises  students  completed  (p  < 
.01).  Clearly,  both  treatments  engaged  the  learners  in  some  deeper  level  thinking. 

However,  the  results  of  this  experiment  were  not  as  distinct  because  the  nature 
of  the  treatments  were  so  similar.  In  this  study,  we  over-compensated  for  problems 
experienced  in  the  previous  study.  The  processing  that  the  two  strategies  required 
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was  fairly  equivalent,  though  slightly  more  difficult  for  the  group  who  completed 
their  own  maps.  That  is,  there  was  not  a  sufficient  difference  between  the 
instructional  strategy  and  the  learning  strategy  used  in  the  study.  The  difference 
in  performance  was  most  obvious  on  the  analogy  questions,  which  students  always 
find  most  difficult.  In  subsequent  research,  the  activities  should  be  more  clearly 
distinguished. 

In  general,  students  performed  better  on  intentional  learning  (items  cued  by 
the  maps)  than  on  incidental  learning.  The  significance  of  this  difference  is  more 
complex  than  it  at  first  seems,  since  few  of  the  test  items  were  directly  represented 
on  the  maps.  Moreover,  with  the  exception  of  the  proximity  items  (the  most  nebulous 
of  the  structural  knowledge  items),  the  mapping  group  generally  outperformed  the 
teacher-provided  group  on  both  intentional  and  incidental  learning.  Thus  the 
more  generative  strategy  seems  to  have  facilitated  both  intentional  and  incidental 
learning  more  than  did  the  more  instructor-supported  strategy.  However,  it  is 
again  important  to  note  that  the  strategies  involve  relatively  similar  degrees  of  in- 
structional support.  Results  of  a  prior  study  by  Jonassen,  Cole  and  Bamford  (1992) 
suggest  that  placing  heavier  cognitive  demands  on  learners  can  be  counter-produc- 
tive. 

Future  Research 

This  study  only  began  to  address  the  many  questions  about  how  best  to  support 
the  acquisition  of  structural  knowledge.  Future  studies  must  address,  for  example, 
(a)  the  relationship  between  individual  learning  characteristics  (e.g.,  verbal  abil- 
ity, and  field  dependence/independence),  (b)  how  much  instructional  support  is  op- 
timal (the  results  of  this  study,  combined  with  those  of  the  previous  study  by 
Jonassen,  Cole,  and  Bamford  (1992),  challenge  the  assumption  that  the  more  tlie 
learner  generates,  the  more  he/she  will  learn),  (c)  what  concepts  to  map,  (d)  how  to 
depict  concepts  (e.g.,  Do  different  types  of  frames  differentially  facilitate  the  ac- 
quisition of  specific  types  of  structural  knowledge?),  (e)  the  role  of  incentive  in  such 
studies  (the  current  study  may  not  have  provided  adequate  incentive;  would  greater 
incentives  increase  the  effects?).  Future  research  with  college-level  students 
should  also  include  a  control  group  who  invoke  their  own  strategies. 
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DISTANCE  EDUCATION: 
A  COST  ANALYSIS 


by 

Judy  I.  Jones 
Michael  Simonson 


INTRODUCTION 


The  concept  of  distance  education  is  enlarging  the  definitions  of  how  students  learn,  where  they  learn, 
and  who  teaches  them  (U.  S.  Congress,  1989).  In  the  past,  distance  education  was  of  greatest  importance 
to  adults.  Recently,  there  have  been  increasing  numbers  of  distance  education  opportunities  offered  to 
high  school  students,  and  even  elementary  schools  have  taken  advantage  of  enrichment  activities 
offered  to  children  from  distant  sites. 

Distance  education  has  become  increasingly  sophisticated  since  the  first  educational  radio  programs. 
Barker  (1989a)  referred  to  distance  education  as  a  "catch  all"  phrase  for  any  form  of  instruction  in 
which  the  learner  and  instructor  were  separated  geographically  and  were  linked  by  telecommunication 
systems  that  permitted  live,  interactive  audio  and /or  video  exchanges. 

Definitions  of  distance  education  usually  include  Keegan's  (1986)  characteristics: 

•  quasi-permanent  separation  of  teacher  and  leamer, 

•  influence  of  an  educational  organization, 

•  use  of  technical  media, 

•  provision  of  two-way  communication,  and 

•  quasi-permanent  absence  of  group  learning. 

Keegan  also  includes  the  following  characteristics  as  ones  that  are  often  part  of  distance  education 
systems: 

•  presence  of  industrialized  features,  and 

•  privatization  of  institutional  learning  (p.  49-50). 

Distance  education  systems  that  use  two-way  interactive  telecommunications  have  the  following 
features.  The  instructor  is  at  an  origination  site  and  students  are  at  a  remote  site;  students  may  be 
present  at  the  origination  site  as  well.  Instruction  is  influenced  by  an  educational  organization-the 
school.  Sites  are  connected  point-to-point  with  fiber,  microwave,  or  twisted  pair  copper  media,  often 
referred  to  as  compressed  video.  This  allows  two-way  interactive  video,  data,  and/or  audio 
communication  and  permits  live  interaction  between  instructor  and  students,  with  instruction  directed  to 
one  or  more  students. 

In  this  study,  the  term-compressed  video"  refers  to  video  signals  that  have  been  digitized  and 
significantly  compressed  so  they  can  be  transmitted  using  traditional  transmission  media,  including 
twisted  pair  copper  wire.  The  terms  fiber  and  microwave  refer  to  the  media  for  transmitting  video  and 
daU  signals,  and  compressed  video  refers  to  a  technique  for  modifying  a  video  signal.  Although  they 
are  not  directly  related,  the  three  terms  (fiber,  microwave,  and  compressed  video)  are  used  in  the  study 
because  they  are  commonly  cited  in  distance  education  literature. 

W»  would  Ilk*  to  AitaowWdjp  the  Office  of  Ttlecommunlcadona  and  the  MedU  Reeourctt  Center  at  Iowa  Stat*  Unlvewlty  for  their  review  of  the 
technical  information  in  thla  monograph. 
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The  trend  in  distance  education  in  the  United  States  is  to  provide  distant  learners  with  live  two-way 
S^lKS?  ^  HdiStanCC  CdUCati0n  iS  de^m  on  interactio"  and  pa^idpat  on  o7  ' 

Today  the  technical  capabilities  exist  for  schools  to  provide  distance  education  opportunities  to 

SHi?*     anCCS u  teChn°'0gy  make  U  P0"1"8  for  sch001  districls  *»  thrfr  curricula 

f  tf  expenences,  the  use  of  technology  also  creates  new  problems.  Technology  is  expensiw  and 

Xd ^emotirs^0"  *"  °'  "**  te,«°«tions  to  offer  courses SSud^s 

sSStSS^I*^  b°ardS' P°,icy-™kers  often  as*  «bout  how  much  distance  education  costs. 
Since  distance  education  systems  require  significant  cost  commitments,  those  responsible  for  making 

Sng^.VeStmentS  mUSt  haVC  reHab,e  d3ta  aVai,ab,C  10  them  10  heIP  theTm  the  deS^ 

Si4?iLStUdy  co<m,PareS.the  C0StS,  of  three  We*  of  transmission  technology.  Two  of  the  technologies 
dehver  two-way,  full  motion  vide6  and  two-way  audio:  (a)  fiber  optics  and  (b)  microwave  The 

^^^t't^  de,iVCrS  ^  3Udi0  and  ^ 

c^pt^^  Listsof 

The  following  research  questions  guided  the  preparation  of  this  report. 

0)  ^chlbUhs?CharaCteriStiCS'  advanta8es'  and  disadvantages  of  the  three  transmission 

(2)  What  does  it  cost  to  put  into  place  a  point-to-point,  distance  education  system  using  fiber 
microwave,  and  compressed  video?  /?      U!,mg  noer' 

(3)  What  are  the  major  cost  considerations  for  the  three  transmission  methods? 

5  wZt^lC0StS       for  distances  of  3  mi,es'  10  mm*  20  miles,  30  miles,  50  miles,  and  75  miles' 
5  What  are  he  recurring  and  maintenance  costs  for  the  transmission  equipment? 
(6)  Is  it  usually  better  to  lease  or  purchase?  V 

S  )Sa!  d°eSJt  C0St  t0  ****  an  interacnve  distance  education  classroom? 
(8)  What  are  the  recurring  and  maintenance  costs  for  the  classroom  equipment? 

^fZhedUCatiH°1 iS  3  Viab,'e  Way  t0  enHch  and  imProve  the  I™"*  of  curriculum.  Technology  can 
link  teachers  and  learners  from  all  educational  levels-elementary,  high  school  and. col  STJTSn^ 

can  be  justified  by  an  increase  in  educational  opportunities  (Giltrow,  1989).  c'assroom.  ">sts 

REVIEW  OF  LITERATURE 

This  review  of  the  literature  summarizes  research  related  to  distance  education  and  key  variables  of 
this  study.  The  information  in  this  section  is  organized  in  the  following  categories  (a)  a  br  ef 
descripuon  of  theories  of  distance  education;  (b)  the  impact  of  distanced  JSrto  ^  Jay  (c  the 
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Distance  Education 

There  are  two  views  on  the  concept  of  distance  education  -  distinctive  and  parallel. 

Distinctive  form.  Many  consider  distance  education  to  be  a  distinctive  educational  form  based  on 
individualized  study.  Many  autonomous  universities  teach  only  to  students  at  a  distance  and  believe 
that  distance  education  is  a  distinct  form  of  education  (Moore,  1988;  Peters,  1988). 

Considerable  effort  has  been  spent  trying  to  formalize  a  theory  specific  to  distance  education.  Keegan 
(1988)  grouped  the  theories  as  follows:  (a)  theories  of  autonomy  and  independence:;  (Wedemeyer,  1981; 
Moore,  1988);  (b)  theory  of  industrialization;  (Peters,  1988);  and  (c)  theories  of  interaction  and 
communication;  (Holmberg,  1977, 1981, 1988a,  1988b;  Baath,  1980;  Sewart,  1988).  Keegan  (1980, 1986) 
analyzed  the  theories  presented  by  Moore,  Peters,  and  Holmberg  to  create  the  descriptive  definition  of 
distance  education  summarized  above.  It  is  the  definition  used  most  widely  today. 

Parallel  form.  A  second  perspective  considers  distance  education  as  parallel  to  traditional  education. 
Garrison  (1989)  asserts  that  distance  education  is  not  unique.  The  techniques  appropriate  for 
traditional  education  systems  and  distance  education  systems  are  the  same.  Many  educators  in  the 
United  States  subscribe  to  the  philosophy  that  distance  education  does  not  differ  from  conventional 
education  in  any  real  structural  sense,  but  is  distinctive  only  in  the  delivery  system  (Zigerell,  1984). 


Distance  Education  Today 

Applications  of  distance  education  have  increased  dramatically  during  the  last  five  years.  Fewer  than 
ten  states  were  promoting  distance  education  in  1987,  while  today  virtually  all  states  have  an  interest 
or  effort  in  distance  education  (U.  S.  Congress,  1989).  Distance  education  programs  exist  in  the  majority 
of  countries  in  the  world.  This  growth  is  primarily  due  to  increases  in  educational  requirements  that 
have  coincided  with  the  expanding  capabilities  and  services  of  the  telecommunications  industry 
(Ciltrow,  1989).  The  quality  of  distance  learning  has  been  recognized  with  increasing  respect  and 
credibility  (Turnbull,  1988).  Distance  education  has  been  seen  as  a  viable  and  cost-effective  way  to 
meet  the  challenges  of  teacher  shortages,  low  student  enrollments,  and  decreased  funding. 

Jefferson  and  Moore  (1990)  predict  that  all  rural  schools  will  have  distance  education  service  by  1995. 
A  UCLA  study  predicts  that,  by  the  year  2010, 50  percent  of  the  educational  instruction  in  the  United 
States  will  be  "mediated  education"  (cited  in  Pelton,  1990). 

The  success  of  interactive  distance  education  classes  is  well  documented.  Studies  have  supported  the 
following  conclusions: 

•  Students  involved  in  distance  education  classes  have  reported  positive  feelings  about  their 
experiences  (Barker,  1989b;  Catchpole,  1988;  Kitchen  4  Kitchen,  1988;  Nelson,  1985;  Nelson, 
Cvancara  &  PeterM989;  Pirrong  &  Lathen,  1990;  U.  S.  Congress,  1989). 

•  After  teaching  in  an  interactive  distance  education  system,  faculty  have  had  positive  responses 
about  the  experience  (Barker,  1989b;  Bowman,  1986;  Hobbs,  1990;  Kitchen,  1987;  Randall  &  Valdez, 
1988;  U.S. Congress,  1989). 

•  Student  achievement  in  interactive  distance  education  classes  has  been  as  good  or  better  as  that  of 
students  learning  from  traditional  teaching  methods  (Batey  Sc  Cowell,  1986;  Hobbs,  1990;  Kabat  & 
Friedel,  1990;  Minnesota  State  Department  of  Education,  1990;  Pirrong  &  Lathen,  1990;  Randall  & 
Valdez,  1988;  U.S.  Congress,  1989). 
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The  Economics  of  Distance  Education 


Economic  implications  of  using  distance  education  methods  can  be  generalized  as  follows:  (a)  significant 
costs  are  incurred  irrespective  of  student  numbers,  (b)  transmission  and  production  costs  are  n  eh  for 
the  Z,'£  STTTi T*iSt™Ce  edUCaH°n  SyStem  C0StS  are  recur^S  costs  and  vary  according  to 

of  a  cour^  (d) ZtT       ™  ^  be  35  fixed  C0S'S  and  amort^  over  tX 

of  a  course,  (d)  from  an  econom.c  point  of  view,  investment  where  student  numbers  are  small  is  normillv 

ISLZT?;"*  (f>  administrative  f-ctions  «  —  clearly  differenHa^tom  SSc SET* 
functions  in  distance  education  systems;  distance  systems  are  more  complex  (Rumble,  1982,  p.  119) 

2Uin«  nf  frSt  Significant  factors  when  Plannin8  a  Stance  education  system  is  the  cost.  A  large 
portion  of  the  expenses  are  start-up  costs,  which  in  effect,  can  be  the  equivalent  of  five  years  worth  of 

S tn  "fS- J°  thCSe  00515  bef°re  snrollment  revenues  are  collected  is  unsettling 

Qltrow  1989),  and  ,s  only  acceptable  if  costs  can  be  amortized  over  a  long  period  of  S  Perfaton 
982).  Factors  ha  affect  costs  of  establishing  a  system  are  (a)  type  of  technology,  (b)  distance  and 

topography  (c)  exisHng  and  available  technology,  (d)  possible  partnerships,  (efengmeX 

S^XTSnt6 S^ATt?  ^JT^W^mM  ^a«"|and  maintenance 
cosis  are  also  significant  (Shobe,  1986).  These  costs,  sometimes  referred  to  as  recurrine  costs  and  fixed 

SlStSL"???      repa'r'  fCeS'  mai"^a"ce  fees,  and  lease  lee^SSSS^ 

required  to  establish  a  system  is  often  seriously  underestimated  (Rumble,  1986b).  Obviously  S,nnel 
and  curriculum  costs  to  train  instructors  and  revise  materials  are  considerable,  too  ^ 

In  general,  three  factors  influence  the  economics  of  distance  education:  choice  of  media  size  nr  „f 

SKS."  t  iouambrr  of  students  (Batey  &  Cowe!1' 1986<-  Rumb,e-  1982>"  ^S^Smb^l2£°f 

that  distance  learning  is  not  necessarily  a  cheap  way  of  teaching"  (p.  137),  the  large  caoital 

ZZTZ  £Zf  inHC°S?ffiCienCy  " therC      SUffkient  ^  °f  ^dentr8Mar£u  (1990) 
maintains  that  distance  education  programs  are  predominantly  self-sufficient. 

'S  availab'fut0  suPP°rt  the  conclusion  that  distance  education  is  effective  when 
iS™SU\?  buy  achie,vement<  by  a«itud<*  of  students  and  teachers,  and  by  c^st 
cffectiveness.  Generally,  the  evidence  has  indicated  that  distance  education  "works." 

Distance  Education  and  Technology 

Telecommunications  is  defined  as  communicating  over  a  distance.  Rapid  advances  in  technoloev 
derations  of  these  technologies,  and  decreases  in  costs  have  m.de^IecomS.ton^&ive  to 

JSSST^Jr  ^StCmS      °ften  refe,Ted  t0  by  the  medium' the  ^  of  ^formation  (signal) 
r.nsmittea  and  the  direction  flow.  The  type  of  information  include,  audio,  video  anS  datoSmah 
he  media  for  communication  are  usually  radio  waves  through  the  air,  electroniclmnSlses  oveT 

r  made  of  8,ass  or  p,astic  silicon- z  KSSK  on, 

One-way  communications  are  usually  referred  to  as  broadcast  systems  and  are  usually  "toiwlown" 

™ nH  S?      y       bC  transm,tted  t0  an  opting  satellite  and  then  broadcast  to  receivers  on  the 
fZJ       ,  'S  un,form,ly ,n  the  Programs  offered  and  usually  no  direct  interaction  Th£c  ontwav 
referred  t0  35  ^^-^  TelevisionLadcasts  are  example^TonTw^ 

In  a  two-way  system,  points  are  linked  throueh  a  network  and  tend  to  encourage  "bottom  up"  and 
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lateral  communications.  They  often  involve  live,  real-time  interaction  between  individuals  at  two  or 
more  places.  This  communication,  often  referred  to  as  point-to-point  or  narrowcasting,  is  directed  to  a 
specific  audience. 

Two-way  interactive  television  (1TV)  and  distance  education.  The  term  ITV  originally  referred  to 
instructional  television,  but  today  has  become  synonymous  with  interactive  television.  Today's  ITV 
technologies  allow  for  live,  two-way  communication  between  sites.  In  the  ITV  classroom,  the  students 
and  teacher  see  and  hear  one  another.  ITV  has  been  shown  to  be  comparable  to  the  traditional  method 
of  instruction,  both  in  terms  or  teacher  and  student  acceptance  and  in  achievement  (Hughes,  1988).  The 
Office  of  Technology  Assessment  reported  that  television-based  interacti*'?  instruction  is  the  distance 
education  format  that  most  closely  resembles  the  traditional  classroom;  this  has  been  the  format  of 
choice  for  many  U.S.  distance  education  systems  (U.  S.  Congress,  1989). 

Bates  (1988),  Feasley  (1983),  and  Cray  (1988)  state  there  is  a  clear  movement  away  from  using 
broadcast  methods  for  distance  learning  systems.  "In  the  1990s,  it  is  no  longer  a  question  of  whether  ITV 
is  going  to  become  a  major  factor  in  the  delivery  of  education.  ITV  is  here,  and  there  is  obvious  value  in 
its  use  as  a  delivery  tool"  (Moore  k  McLaughlin,  1992,  p.  76). 


Transmission  Systems 

Since  it  is  possible  to  discuss  distance  education  systems  from  a  number  of  perspectives,  this  monograph 
examines  the  costs  for  establishing  a  system  by  studying  three  forms  of  technology  that  are  often  used  to 
characterize  particular  systems:  fiber-based  systems,  microwave  systems,  and  compressed  video 
systems  that  use  existing  transmission  networks.  As  was  stated  earlier,  the  three  are  not  directly 
comparable.  However,  when  distance  educators  discuss  systems,  they  often  categorize  them  in  this 
manner. 

Fiber  Optic  Systems 

Fiber  optics  is  one  of  the  newest  two-way,  interactive  technologies.  The  fiber,  made  of  glass  or  plastic, 
transmits  light  signals  instead  of  electrical  signals.  An  optical  fiber  consists  of  an  inner  cylinder  called 
the  core,  surrounded  by  a  cylindrical  shell  of  glass  or  plastic  called  the  cladding.  The  cladding  layer 
keeps  light  from  leaking  out.  An  outside  coating  provides  protection  against  the  elements.  Light 
travels  in  straight  lines,  but  optical  fibers  guide  light  around  corners.  The  number  of  fibers  is  unlim- 
ited, creating  virtually  an  unlimited  capacity.  Optical  cables  are  capable  of  transmitting  far  more 
information  than  coaxial  cables  of  the  same  size. 

Equipment  required  for  a  fiber  system.  The  main  components  of  a  digital  fiber  system  are:  (a) 
multiplexor;  (b)  codec;  (c)  optical  transmitter;  (d)  optical  receiver  or  photodetector;  (e)  fiber  cable;  and 
(0  repeaters. 

The  multiplexor  converts  the  signal  to/from  an  electrical  signal.  The  codec  changes  the  signal  to 
digital.  The  optical  twmsmitter  converts  the  signal  to  an  optical  signal,  and  the  receiver  reconverts  the 
optical  signal.  Transmitters  are  of  two  types,  lasers  (ILD)  or  light  emitting  diodes  (LED).  The 
receivers  are  either  positive-intrinsic-negative  (PIN)  or  avalanche  photodiode  (ADP).  Generally 
speaking,  the  ADP  is  used  for  systems  greater  than  100  km  (62  miles)  and  PINs  are  preferred  for  shorter 
distances.  The  repeater  is  a  signal  amplifi-cation  device  often  used  along  cables  to  extend  transmission 
distances.  The  fiber  cable  carries  the  optical  signal.  Single  mode  and  graded  index  fiber  is  best  suited 
for  long  distances.  Since  a  codec  is  needed  only  for  digital  transmission,  an  analog  system  would 
eliminate  need  for  the  codec.  A  modulator/demodulator  would  replace  the  multiplexor  unit; 
edditional  amplifiers  would  be  necessary.  Life  expectancy  of  this  equipment  is  20-25  years. 
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Advantages  of  a  fiber  system. 

•  The  large  bandwidth  of  fiber  allows  audio,  video,  and  data  to  be  combined  on  one  line,  resulting  in 
a  lower  cost  per  channel.  5 

•  Fiber  permits  full  motion  video  transmissions. 

•  The  low  attenuation  rate  allows  for  transmissions  over  long  distances  without  distortion. 

•  The  large  capacity  for  channels  means  the  system  can  be  easily  expanded 

"    metol  ducting™  ****  P°SSibility  of  taPPin&  eavesdropping,  jamming  or 

•  The  small,  lightweight  cables  are  easy  to  handle  and  install. 

•  Fiber  is  unaffected  by  weather,  corrosive  liquid,  or  gas. 

•  Fiber  is  unaffected  by  electromagnetic  currents,  static  interference,  electric  motors,  fluorescent 
lights,  and  radiation. 

•  Durable  fiber  results  in  low  maintenance  costs. 

Disadvantages  of  a  fiber  system,. 

•  Systems  require  high  start-up  costs. 

•  Special  tools  and  tests  are  needed  to  install  the  fiber. 

•  Repai  rs  can  be  time  consu  mi  ng  and  cost  ly . 

•  Light  sources  have  limited  lifetimes  and  associated  system  reliability  problems 

•  Expansion  is  expensive  if  no  fiber  exists. 

•  Right-of-way  costs  for  placing  cables  in  the  ground  can  be  costly. 

s^y^CT^-^r^?6  f°"0Wing  S™*5  haVC  rep°rted  on  succcssful  use  of  «ber-based 
989  Ti'trhpn  io«7~^  ;  '*  Ed.u«t.onal  Telecommunications  Cooperative,  Minnesota  (Ciltrow, 
I  a  m  I  V  ?d  >  n  &  K,tChen'  1988;  Lanier'  1986);  (b>  SHARE-ED  Oklahoma  Panhandle 
Video  Network  (Barker,  1989b;  Currer,  1991);  (c)  Mississippi  2000  (Currer,  1991);  (d)  Bergen  County 
New  ersey  (Da  ey,  1991);  (e)  FOCIS-Fiber  Optics  Communication  Instruction  System^Des  Momes  ' 
Iowa  (Ostendorf,  1989a;  Schoenenberger,  W.,  personal  communication,  April  14 ffi 

Opinion?  abovt  fiber  sterns.  "It  is  possible  to  confidently  claim  that  this  [fiber  optics]  is  one  of  the 
more  important  technological  advances  of  the  past  20  years"  (Ciltrow,  1989,  p.  55).  Fiber  optics  is 
increasingly  being  chosen  as  the  preferred  method  of  transmission  by  Minnesota  schools  because  of  the 
possibility  of  future  expandability  (Minnesota  State  Department  of  Education,  1990).  Fiber  lends  itself 
well  to  two-way,  interactive  television  because  of  its  large  channel  capacity  (Kitchen,  1987).  The  cost 
of  fiber  cable  .s  widely  expected  to  fall  below  coaxial  or  copper  cables  in  the  early  1990s  (U  S 
Congress,  1989);  it  will  be  competitive  with  microwave  and  coaxial  cable  (Kitchen,  1987)  The 

^Z!r£ rmaLCe  rati°,  fu  fiber  continues  t0  imProve"  In  'he  last  ten  years,  fiber  transmission  has 
become  the  medium  of  cho:ce  m  telephone  applications  (Szentesi,  1991).  Currer  (1991)  reported  that 
digital  fiber  gives  the  user  the  most  benefit  for  the  money  spent.  Barker  (1989b)  believesThat  fiber 
o.  fen  the  best  audio  transmission  quality,  while  Lanier  (1986)  suggests  that  fiber  is  superior  to 
microwave,  1TFS,  androaxial  because  of  its  higher  image  quality. 

n^'^f°r^rSVStfniS  .Ihe  °ffiCC  °f  Technol°8y  Assessment  reported  that  the  equipment 
needed  to  connect  one  site  to  a  fiber  network  would  cost  approximately  $40,000.  Fiber  for  a  134  mile 
network  would  cost  approximately  $8,955  per  mile.  The  receiving  site  would  need  $40,000  worth  of 
equipment,  also  (U.  S.  Congress,  1989,  p.  174).  u 

XJryST  C°SH  aTf  u°ne  PerCem  °f  SyStCm  C0St  1**  year  (National  Sc"00'  Board  Associ-ation, 
1989).  Other  vendors  feel  that  up  to  13%  of  the  cost  of  a  system  should  be  reserved  for  maintenance 

ex^sXetd  Z ^lTntCnanC0;  h0Wl,VCr'  "  lhC  *"  iS  aCddenta"y       ^irs  are  very 
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Microwave  Systems 

Microwave  signals  are  transmitted  electromagnetically  through  the  air.  Microwave  is  similar  to 
standard  broadcasting,  except  microwave  systems  use  much  higher  frequencies  and  are  point-to-point. 
Each  tower  in  a  microwave  relay  system  picks  up  the  signal  sent  to  it,  amplifies  the  signal,  and 
retransmits  it  to  the  next  line-of-sight  tower  on  the  way  to  the  destination  point.  Prior  to  the 
development  of  satellite  communication,  microwave  transmission  represented  the  only  reliable  form  of 
intercity  and  coast-to-coast  video  communication  (Hart,  1986). 

"Long  haul"  systems  use  a  number  of  repeaters  and  cover  hundreds  of  miles.  "Short  haul"  systems  are 
used  where  traffic  loads  are  relatively  light,  or  where  the  length  of  the  route  is  short.  The  typical 
range  of  5  to  15  miles  is  suitable  for  local  communication  between  two  schools.  The  distance  between 
repeaters  depends  upon  (a)  topography,  (b)  antenna  size,  (c)  transmitter  power,  and  (d)  receiver 
sensitivity.  A  good  rule  of  thumb  is  to  consider  microwave  if  two  sites  are  more  than  one-half  mile  but 
less  than  20  miles  apart.  A  good  use  of  a  microwa  ve  system  is  to  link  two  buildings  in  the  same 
metropolitan  area.  Short  haul  systems  are  relatively  simple  to  construct  and  operate,  and  do  not 
require  regulatory  agency  approval  or  right-of-way  clearance  (Ostendorf,  1989a). 

Equipment  required  for  a  microwave  system.  The  main  components  of  a  microwave  system  are  (a)  tower, 
(b)  antenna,  (c)  antenna  feed  line,  (d)  transmitter /receiver,  (e)  modulator/multi-plexor,  and  (f)  power 
unit 

Advantages  of  a  microwave  system. 

•  Mir  jwave  permits  the  transmission  of  full  motion  video. 

•  There  is  control  over  who  receives  the  signal. 

•  Audio  and  video  signals  are  of  excellent  quality. 

•  Numerous  data  and  audio  signals  can  be  transmitted  along  with  video  channels. 

•  A  properly  designed  system  can  have  99%  reliability  (Todd  Communications,  1992). 

•  No  right-of-ways  are  required. 

•  Maintenance  costs  are  low. 

Disadvantages  of  a  microwave  system. 

•  Systems  require  high  start-up  costs. 

•  Transmissions  are  affected  by  weather,  especially  fog,  rain,  and  lightning. 

•  Atmospheric  disturbances  can  cause  fading  of  signals. 

•  Equipment  can  fail  and  power  outages  can  occur. 

•  Adding  channels  to  a  system  is  not  easy. 

•  Thirty  miles  is  the  maximum  distance  between  towers. 

•  An  FCC  license  i^required. 

•  A  limited  number  of  frequencies  are  available. 

•  Towers  are  almost  always  needed. 

•  Terrain  extremes  can  increase  equipment  and  tower  costs. 

•  Special  building  permits  may  be  needed. 

Successful  users  of  microwave  systems.  The  following  groups  have  reported  on  their  successful 
microwave  systems:  (a)  WHETS-The  Washington  Higher  Education  Telecommunication  System, 
Washington  State  University  (Nelson,  Cvancara  ic  Peters,  1989;  Oaks,  1986);  (b)  TWIT-Two-Way 
Instructional  Television  at  Morning  Sun,  Iowa  (Nelson,  1985);  (c)  TIE-Televised  Interactive  Education 
Eastern  Iowa  Community  College  District  (Kabat  &  Friedel,  1990,  Wallin,  1990);  (d)  TACER-Texas 
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Association  for  Graduate  Education  and  Research  at  Dallas-Ft  Worth,  University  of  Texas  at  Dallas 
(Hart,  1986);  (e)  KTS--Kirkwood  Telecommunications  System  at  Kirkwood  Community  College  Cedar 
Rapids,  Iowa  (Hart,  1986;  Hudspeth  &  Brey,  1986);  (0  KIDS-Knowledge  Interactive  Distribut'ion 
System,  Minnesota  (Descy,  1991). 

general  costs  for  microwave  systems.  The  estimated  costs  for  a  duplex  microwave  system  include 
$40,000-$65,000,  plus  towers,  which  range  from  $25,000-$75,000  each.  Short  haul  systems  may  cost 
$35,000,  plus  towers,  which  range  from  $5,000-$50,000. 

Maintenance  costs  average  three  to  five  percent  of  the  system  cost  per  year  (Kitchen  &  Kitchen,  1988- 
National  School  Board  Association,  1989;  U.S.  Congress,  1989).  These  costs  include  equipment  service 
and  repair  and  monthly  maintenance  fees. 

Compressed  Video  System? 

Compressed  video  is  a  name  routinely,  and  somewhat  incorrectly,  used  to  refer  to  digital  video,  audio, 
and  data  signals  that  are  processed  to  reduce  the  amount  of  information  in  order  to  minimize  the 
bandwidth  required.  It  is  important  to  note  that  compressed  video  refers  to  a  technique  for  reducing  the 
amount  of  information  transmitted  between  two  sites.  A  "compressed"  signal  can  be  carried  by  fiber  or 
microwave,  but  since  it  has  been  compressed,  traditional,  inexpensive,  and  readily  available  copper 
wires  can  also  be  used.  In  other  words,  the  installation  of  a  compressed  video  system  is  often  simplified 
because  existing  communications  carriers  can  be  used. 

Compressed  video  is  sent  over  fiber,  satellite,  microwave,  and  most  often,  high  capacity  digital  service 
lines.  In  compressed  video  transmissions,  redundant  information  is  removed.  For  example,  if  the 
background  does  not  change  for  several  seconds,  this  information  is  sent  only  once  and  remains  in 
memory.  Video  compression  is  achieved  by  sacrificing  small  amounts  of  color,  motion,  or  resolution 
information.  Although  there  is  some  loss  of  quality,  compressed  video  is  considered  a  viable 
alternative  for  many  educational  situations. 

Equipment  required  for  a  compressed  viHm  «ych.m  Each  system  site  must  have  (a)  codec-  (b) 
transmission  line;  and  (c)  interface  unit.  The  codec  converts  the  analog  signal  to  digital  format  and 
compresses  the  signal.  The  channel  service  unit  (CSU)  is  the  interface  between  the  end  user  and  the 
telephone  line.  The  CSU  works  in  conjunction  with  the  data  service  unit  (DSU),  which  translates  and 
controls  the  signal.  This  unit  functions  similarly  to  a  modem  (modulator/demodulator). 

T  channels  refer  to  a  band  of  high  speed,  high  capacity  circuits  devised  to  carry  digital  voice  audio 
and  data  signals.  Tl  facilities  and  DS1  signals  are  the  technology  most  often  used  for  compressed  video 
systems.  DSl  signals  have  bandwidth  capability  for  compressed  video  only;  they  do  not  have  the 
necessary  bandwidth  for  full  motion  video  as  does  DS3.  Tl  facilities  are  often  regular  copper  telephone 
lines.  DSl  transmission  lines  can  be  leased  from  telephone  companies. 

frame  speed-  Compressed  video  is  available  at  30  frames  per  second  at  384  kb/s  and  1.544  Mb/s  The  P  x 
64  standard  of  30  frames  per  second  is  not  the  same  30  frames  per  second  of  full  motion  video.  The  P  x  64 
standard  rates  pixels  per  screen,  while  the  National  Television  Standards  Committee  (NTSC)  video 
signal  dees  not  have  pixels.  The  P  x  64  thirty  frames  per  second  means  the  picture  is  refreshed  or 
updated  every  30  seconds.  Differences  in  picture  quality  can  exist  within  this  standard.  Different 
vendors'  codecs  refresh  different  elements  such  as  color,  gray  scale,  or  motion, 

Advantages  of  a  compressed  video  system,. 

•  The  equipment  is  easy  to  install  and  use  and  little  training  is  required. 

•  Compressed  video  requires  less  b.indwidth  and  therefore  it  costs  less  to  transmit. 

•  Compressed  video  can  have  a  pnee  advantage  over  fiber  or  microwave  in  long  distance 
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installations  or  systems  in  hostile  terrain. 

•  Compressed  video  interfaces  with  existing  systems  and  is  easily  upgradable  (Todd 
Communications,  1992). 

Disadvantages  of  a  compressed  video  system. 

•  Systems  have  high  start-up  costs. 

•  Motion  can  become  jerky. 

•  Color  may  be  substandard  and  picture  quality  may  be  poor. 

•  There  is  dependence  upon  the  lease-lines  of  the  utility  supplier. 

•  Compressed  video  systems  do  not  transmit  NTSC,  full  motion  video. 

Successful  users  of  compressed  video  systems.  The  following  groups  have  reported  successful  use  of 
compressed  video  systems:  (a)  California  State  University  Campus,  Bakersfield  (Ward,  1990);  (b) 
V.E.l.N.-Video  Education  Interactive  Network,  Wyoming  Center  for  Teaching  and  Learning  at 
Laramie  (Edwin,  Owens  &  Rezabek,  1991);  (c)  Penn  State  (Phillips,  1987);  (d)  IVN-Interactive  Video 
Network,  North  Dakota  (Tykwinski  &  Poulin,  1991);  (e)  University  of  Minnesota  (Kolomeychuk  & 
Peltz,  1991). 

Research  relating  to  compressed  video  systems.    The  Applied  Research  Institute  (cited  in  Keller,  Staab 
&  Stowe,  1989)  found  that  (a)  compression  technologies  have  improved,  (b)  at  the  same  time  that 
signals  have  been  further  compressed  and  equipment  costs  have  dropped,  picture  quality  has  remained 
relatively  good,  (c)  compression  devices  can  now  digitize  and  blend  (multiplex)  signals  so  different 
kinds  of  information  can  be  sent  simultaneously,  (d)  the  hardware  and  software  are  still  expensive  even 
though  prices  have  decreased,  (e)  standards  are  emerging  for  compatibility  among  vendors,  and  (0  use 
of  existing  transmission  lines  make  compressed  video  cost-effective. 

A  study  conducted  in  Japan  reported  that  compressed  video  had  practical  use  for  teaching  at  a  distance 
in  an  interactive  environment  (Wakamatsu  &  Obi,  1990).  Jurasek  (1992)  conducted  a  study  of  the 
effectiveness  of  Iowa  State  University  classes  taught  using  compressed  video  technology.  Students 
displayed  positive  attitudes  toward  instruction.  They  adjusted  rapidly  to  the  compressed  video 
technique  and  felt  the  convenience  of  learning  remotely  far  outweighed  the  effects  of  any  picture 
quality  loss  or  technical  problems. 

General  costs  for  a  compressed  video  system.  Compressed  video  systems  range  in  cost  from  $20,000- 
$300,000  (Ostendorf,  1989a).  A  typical  point-to-point  system  would  cost  approximately  $100,000.  For 
example,  in  North  Dakota,  the  implementation  costs  for  compressed  video  digital  systems  averaged 
$29,502  per  school  in  comparison  to  analog  fiber  systems  that  averaged  $60,706  per  school  (Hobbs, 
1990). 


Distance  Education  Classrooms 

Classrooms  for  1TV  are  usually  one  of  two  types.  The  first  type  is  a  production  studio,  comparable  to  a 
television  studio.  Production  studios  require  special  lighting,  a  control  room,  and  a  crew.  The  second 
type  of  distance  education  facility  is  the  electronic  classroom.  In  this  classroom,  there  is  no  control 
room;  the  technology  is  operated  by  the  instructor.  The  classroom  is  both  a  television  studio  and  a 
learning  environment.  The  concept  is  to  make  the  technology  as  transparent  as  possible  and  to  make  the 
teaching  site  appear  to  be  a  classroom,  not  a  studib- 

Characteristics 

Various  examples  of  distance  education  classrooms  are  discussed  in  the  literature.  The  following  are 
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characteristics  that  are  described  as  important  considerations  for  planning  for  the  installation  of  a 
distance  education  classroom. 

Sj££.  The  classroom  can  be  a  conventional  school  room,  which  typically  is  about  25  feet  x  35  feet. 
Converted  classrooms  can  be  used  as  long  as  acoustic,  lighting,  and  other  needs  are  met  (Hudspeth  k 
Brey,  1986).  Hughes  (1988)  stated  that  a  distance  education  classroom  should  be  longer  than  it  is  wide. 

Soundproofing.  Soundproofing  may  be  necessary  to  exclude  exterior  noise  and  to  reduce  classroom  noise 
Since  sound  can  enter  any-where  air  can  enter,  all  holes  and  gaps  in  or  between  walls,  floors,  and 
overhead  structures  should  be  filled  and  sealed.  Doors  should  be  of  solid  construction  and  equipped 
with  floor  sweeps  and  weather  stripping  (Price,  1991).  Windows  should  be  double  pane,  if  possible. 
Walls,  floors,  and  ceilings  should  be  sound  resistant  and  constructed  of  dense  materials  such  as  concrete 
solid  masonry,  or  double  layers  of  gypsum  board  (Hudspeth  &  Brey,  1986;  Price,  1991). 

Interior  noise  is  generated  by  ventilation  systems,  fluorescent  lights,  television  monitors,  furniture  and 
other  equipment.  Ventilation  systems  should  be  run  constantly  at  a  low  pressure,  if  possible.  Systems 
that  cycle  on  and  off  are  very  distracting  and  may  require  users  to  make  constant  audio  adjustments.  One 
way  to  test  a  potential  distance  education  room  for  external  and  internal  sound  disturbances  is  to  tape 
record  the  room  when  it  is  not  in  use.  This  tape  will  indicate  outside  noise  disturbances  and  distractions 
from  such  things  as  ventilation  and  lights  within  the  room. 

QSSSK.  Esthetic  considerations  for  the  classroom  are  fairly  subjective.  There  are,  however,  particular 
items  that  do  affect  quality.  Background  color  is  particularly  important  for  two-way  video  systems- 
there  should  be  no  complex  patterns.  All  surface  finishes  should  be  non-glare.  Materials  such  as 
chrome,  glass,  and  shiny  plastics  create  distracting  glare  that  can  be  reflected  onto  monitor  screens. 

Lighting.  Television  is  best  viewed  in  normal  light  or  in  a  slightly  dimmed  room.  Natural  light  from 
windows  and  skylights  is  usually  too  bright  for  television  monitors  and  can  cause  glare  on  screens.  This 
can  be  controlled  by  draping  or  facing  the  monitors  away  from  windows.  However,  television  cameras 
require  light  to  capture  images  and  usually  normal  room  lighting  is  adequate.  For  most  purposes 
common  fluorescent  light  fixtures  provide  adequate  and  economical  lighting.  In  the  typicalclassroom, 
no  special  lighting  is  needed  (Ostendorf,  1989a). 

gagsrppm  arrangement.  Two-way  video  classrooms  have  few  possible  arrangements.  Participants  must 
be  seated  in  relation  to  cameras  and  microphones,  thus  limiting  the  configuration  of  the  room  and  the 
size  of  groups.  Students  should  be  seated  so  they  can  see  each  other  and  not  have  to  turn  to  face  the 
camera.  Monitors  should  be  placed  in  the  front  and  possibly  rear  of  the  room  so  participants  and 
instructors  can  maintain  continuous  visual  contact  (Fink  Sc  Tsujimura,  1991). 

It  is  best  to  avoid  placing  monitors  in  a  corner;  this  can  diminish  the  importance  of  the  material  being 
presented  and  can  create  awkward  viewing  angles  (Price,  1991).  A  viewer  should  not  have  to  look  up  at 
an  angle  greater  than  30  degrees  (Price,  1991;  Wood  &  Wylie,  1977).  Looking  upwards  at  a  sharp  angle 
or  long  periods  can  be  tiring  and  uncomfortable.  Monitors  should  be  ceiling  or  wall  mounted,  four  to  six 
feet  off  the  floor,  if  possible  (Wood  &  Wylie,  1977).  Remote  site  monitors  should  simulate  the  eye 
level  of  the  instructor.  7 

There  should  be  comfortable  seating  for  students  and  teacher  (Minnesota  State  Department  of 
Education,  1988).  Individual  comfort  and  appropriate  style  should  be  the  highest  priority.  For 
extensive  note  taking  or  working  with  materials,  learners  should  be  provided  with  20-24  inch  wide 
tables. 

Optimum  class  size  is  15  to  25  students  (Barker,  1989b;  Bowman,  1986).  By  keeping  classes  small, 
students  will  interact  more  readily  and  use  the  system  more  fully  (Lanier,  1986). 
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Teacher  station.  An  important  element  of  the  system  is  the  teacher  station.  There  are  various  forms- 
table,  desk,  lectern,  or  special  built  podium.  The  teacher  station  always  faces  student  seating.  Putting 
this  station  on  a  riser  increases  visibility  (Minnesota  State  Depart-ment  of  Education,  1990). 

It  is  desirable  for  the  teacher  to  have  full  control  of  cameras,  lights,  monitors,  and  various  auxiliary 
equipment.  If  the  instructor  controls  equipment,  there  is  no  need  for  extra  technical  personnel.  This 
promotes  an  instructional  format  like  the  traditional  classroom;  therefore,  teachers  do  not  need  to 
completely  change  teaching  strategies  (Currer,  1991;  Greenwood  &  McDevitt,  1987;  Minnesota  State 
Department  of  Education,  1990). 

Equipment.  There  should  be  standardization  of  equipment  and  installation,  so  in  case  of  breakdowns, 
spare  parts  or  items  are  interchangeable.  Identical  equipment  makes  it  possible  to  verbally  instruct 
someone  on  how  to  adjust  equipment  at  remote  sites,  and  with  identical  equipment,  either  site  can  serve 
as  the  origination  site  (Minnesota  State  Depart-ment  of  Education,  1988;  Randall  &  Valdez,  1988). 

•  Microphones.  Sound  quality  is  the  most  problematic  aspect  of  distance  education  systems;  it  is  the 
most  often  reported  technical  complaint  (Bowman,  1986;  Descy,  1991;  Fink  &  Tsujimura,  1991; 
Hobbs,  1990;  Price,  1991).  Each  component  of  the  audio  system  should  be  of  highest  quality 
(Ostendorf,  1989b;  Price,  1991).  Directional  or  cardioid  microphones  are  best. 

Two  problems  commonly  occur  in  sound  systems,  feedback  and  echo.  Feedback  is  caused  when 
microphones  pick  up  sound  from  the  speakers,  producing  overamplification  and  a  piercing  squeal 
from  the  speaker.  Any  time  open  microphones  and  open  speakers  are  in  the  same  room,  feedback  is 
a  potential  problem.  This  can  be  alleviated  by  separating  the  microphones  and  speakers. 

Echoes  in  the  room  are  caused  by  sound  reflections  off  smooth  surfaces  such  as  hard  walls,  floors, 
and  ceilings.  The  shape  of  the  room,  the  floor,  ceiling,  and  walls  all  affect  reflection.  An 
acoustically  treated  room  will  not  have  echoes.  To  reduce  echoes,  acoustical  ceiling  tile  and  wall 
covering  should  be  considered.  Carpet  is  the  best  and  least  expensive  sound  absorber  to  improve 
acoustics  in  distance  education  classrooms  (Hughes,  1988;  Minnesota  State  Department  of 
Education,  1988;  Price,  1991).    Television  does  not  add  to  acousti-cal  difficulties  already  existing 
n  a  room,  but  it  does  make  them  more  acute.  Acoustical  problems  in  the  class  can  negatively 
impact  on  learning  (Smith,  1961). 

•  Cameras.  Most  interactive  classrooms  have  three  cameras.  One  is  pointed  at  the  students,  the 
second  at  the  teacher,  and  the  third  is  at  the  desk  or  above  the  instructor,  focused  on  graphic 
materials  (Barker,  1989b;  Bowman,  1986;  Currer,  1991;  Lanier,  1986;  Minnesota  State  Department 
of  Education,  1990;  Nelson,  Cvancara  &  Peters,  1989;  Oaks,  1986).  Some  classroom  systems  have  a 
fourth  camera  pointed  at  students  (Fink  &  Tsujimura,  1991). 

Remote  zoom  and  auto  focus  controls  for  cameras  are  recommended  (Hughes,  1988;  Minnesota  State 
Department  of  Education,  1988;  Oaks,  1986).  Remote  tilting  and  panning  can  also  be  added  as  extra 
features. 

•  Monitors.  Monitors  let  the  instructor  and  origination  site  students  view  students  at  the  remote  site. 
Monitors  also  let  the  instructor  view  what  is  being  transmitted.  Monitors  at  the  remote  site  show 
what  is  being  transmitted  from  the  origination  site. 

Most  systems  use  21-25  inch  monitors  (Barker,  1989b;  Minnesota  State  Department  of  Education, 
1988;  Smith,  1961).  Others  use  large  screen  projectors  (Fink  tc  Tsujimura,  1991;  Greenwood  k 
McDevitt,  1987).  Some  systems  use  a  split  screen  to  present  the  view  from  two  cameras.  Hughes 
(1988)  reported  that  the  split  screen  makes  remote  sites  even  smaller  and  seemingly  more  remote. 
He  suggested  using  an  additional  25  inch  monitor  rather  than  splitting  the  screen. 
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Visual  presenter.  This  device  functions  like  an  overhead  projector.  It  is  located  at  the  teacher 
station,  normally  to  the  right  of  the  instructor.  A  color  camera  in  the  device  projects  photographs, 
charts,  maps,  and  three  dimensional  objects.  The  camera  has  the  ability  to  zoom  in  on  the 
platform  where  materials  are  placed  and  functions  similarly  to  the  overhead  camera. 

Telephone.  A  telephone  provides  for  alternative  communication  in  case  of  emergency  or  equipment 
failure  and  allows  for  a  direct  link  to  the  remote  site  (Minnesota  State  Department  of  Education, 
1988;  Randall  &  Valdez,  1988).  It  also  allows  for  student/ teacher  one-on-one, 
semiprivate/private  conversations. 

Facsimile  machine-  A  facsimile  transmits  printed  copy  from  classroom  to  classroom  quickly.  It  can 
be  used  to  supply  immediate  feedback  for  tests  and  corrected  assignments  (Descy,  1991;  Minnesota 
State  Department  of  Education,  1988).  A  facsimile  may  not  be  necessary  if  there  is  a  dependable 
delivery  service  or  a  traveling  teacher  or  administrator. 

VidePWSSette  recorder  (VCR).  The  VCR  is  used  to  record  class  sessions.  This  recording  can  be  used 
by  the  student  who  has  been  absent,  as  a  backup  in  case  of  technical  difficulty,  to  provide  a  record 
of  distracting  student  behavior,  or  so  that  the  instructor  can  review  for  self-critique  (Minnesota 
State  Department  of  Education,  1988;  Randall  &  Valdez,  1988).    Special  class  sessions  can  be 
recorded  for  use  in  the  future,  also. 

Auxiliary  equipment-  In-class  presenta-tions  may  require  the  use  of  a  videodisk  player,  a 
viJeocassette  player,  a  tape  recorder,  or  a  record  player.  This  equipment  can  be  located  in  the 
room  on  a  cart  and  plugged  into  the  system  as  an  auxiliary  input.  Films  and  slides  may  also  be 
shown  as  an  in-class  presentation  by  projecting  the  film  or  slides  onto  a  projection  screen  and 
focusing  either  the  teacher,  student,  or  visual  presenter  camera  on  the  screen.  If  there  are  many 
slide  and  film  presentations  planned,  slide/film  to  video  converters  should  be  purchased. 

Computers-  Use  of  computers  as  an  auxiliary  input  requires  special  equip-ment.  Computer  monitors 
have  different  resolution  capabilities  and  different  scan  rates  than  television  monitors.  Special 
equipment  is  needed  to  interface  the  two  types  of  video.  Output  from  high  resolution  computers 
must  be  converted  down  to  the  NTSC  video  standard,  or  multiscan  monitors  must  be  used 


METHODOLOGY 


This  cost  analysis  identified  costs  required  to  design,  build,  and  install  a  distance  education  system 
using  fiber,  microwave,  and  compressed  video.  Cost  analysis  was  defined  by  Perraton  (1982)  as  the 
PirQ0^SS,UfSed  1°  discover  3,1  the  costs  involved  in  a  particular  activity.  Adams,  Hankins,  and  Schroeder 
(1978)  defined  cost  analysis  as  any  manipulation  of  cost  data  that  provides  relevant  information  for 
those  who  make  decisions.  A  common  method  is  simply  "a  direct  comparison  method  between  or  amonc 
the  costs  of  specific  decision  alternatives,  e.g.,  a  make  or  buy  decision"  (p.  24). 


Data  Collection 


Data  were  collected  using  face-to-face,  telephone,  and  written  interviews.  Questions  were  designed  to 
determine  the  total  costs  for  establishing  a  distance  education  system.  Interviews  were  open-ended  to 
offer  flexibility  and  opportunity  to  pursue  additional  information.  Reliability  was  built  into  the 
interview  process  by  the  use  of  repeated  questions,  rephrased  questions,  and  follow-up  questions.  Also, 
interview  questions  were  asked  of  more  than  one  source. 
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Procedures 

The  first  activity  of  this  project  was  to  obtain  general  information  from  the  literature,  consultants, 
telecommunications  specialists,  governmental  offices  (federal  and  state),  and  vendors.  Next,  specific 
questions  were  prepared.  These  questions  were  designed  to  identify  hardware  components  essential  to 
transmit  and  receive  audio  and  video  signals  between  two  sites. 

Technical  experts  who  had  experience  with  the  transmission  systems  provided  valid  cost  information. 
Also  identified  were  vendors  and  consultants  who  had  experience  with  distance  education  classroom 
equipment  (Appendix). 

Six  hypothetical  pairs  of  sites  of  varying  distances  from  one  another  were  chosen.  Vendors  were 
identified  to  provide  estimates  about  costs  about  transmission  media. 

For  the  classroom  equipment  phase  of  this  study,  technical  experts  aided  in  the  identification  of 
designs,  equipment  items,  and  installation  procedures.  This  process  resulted  in  a  classroom  design  and 
equipment  list.  Vendor  catalogs  and  vendor  quotations  were  used  to  obtain  equipment  prices. 


RESULTS 

Costs  are  summarized  and  categorized  in  two  sections.  The  first  section  deals  with  the  costs  of  fiber, 
microwave,  and  compressed  video  systems.  The  second  section  deals  with  the  interactive  distance 
education  classroom  costs. 

Costs  of  Fiber,  Microwave,  and  Compressed  Video  Systems 

When  setting  up  a  distance  education  transmission  system,  specific  elements  define  costs  for  each 
technology.  These  elements  usually  fall  into  three  areas-feasibility  and  planning,  system  design,  and 
purchase  and  installation. 

Fiber  Systems 

Fiber  optic  systems  consist  of  cables  that  transmit  light  signals  rather  than  electrical  signals. 
Purchasing  and  installing  fiber  systems  is  a  complex  process.  Nearly  95  percent  of  the  interactive 
television  systems  designed  today  use  fiber  optics  in  combination  with  coaxial  cables. 

Fiber  systems  can  be  analog  or  digital.  While  the  fiber  is  identical,  the  termination  equip-ment  is 
different.  Digital  fiber  provides  higher  quality  at  a  distance.  Analog  is  more  appropriate  for  short 
distances.  Analog  fiber  picks  up  noise  over  distances;  digital  fiber  does  not.  An  analog  system  is  cost- 
effective  for  distances  under  30  miles,  but  the  costs  for  an  analog  fiber  system  over  30  miles  may  be 
prohibitive  since  a  repeater  (booster)  is  needed  every  30  miles  (at  a  cost  of  $15,000-$1 8,000).  A  building 
protecting  each  repeater  is  also  required  (at  a  cost  of  $1,500). 

What  does  it  cost  to  put  into  place  a  point-to-point  fiber  system?  The  total  cost  of  a  fiber  system 
depends  on  the  fiber  type,  total  system  design,  and  location.  T.  Crandall,  General  Services,  indicated 
that  fiber  for  state  of  Iowa  projects  cost  approximately  $12,000  per  mile,  based  on  10  miles  (personal 
communication,  June  4, 1992).  H.  Sarrazin,  Tele-Systems,  concurred  that  f*ber  costs  $10,000-$15,000  per 
mile  (personal  communication,  May  21, 1992),  and  W.  Fackler,  Spectra  Associates,  generalized  costs  of 
fiber  at  $15,000-$20,000  per  mile  (personal  communi-cation,  May  29, 1992).  D.  Takkunen,  Todd 
Communications,  estimated  fiber  costs  at  $20,000  per  mile  (personal  communication,  April  27, 1992). 
All  of  these  estimates  were  strictly  for  labor  and  fiber;  end  point  equipment  and  right-of-way  expenses 
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have  not  been  included  in  the  totals  above.  In  metropolitan  areas  where  there  are  right-of-way 
expenses  and  higher  costs  for  labor,  installation  rates  may  be  as  high  as  $70,000  per  mile. 

One  vendor  supplied  budgetary  cost  data  for  fiber  systems  at  approximately  $18,000-522,000  per  mile 
(Appendix).  T.  Crandall,  General  Services,  stated  Iowa  schools  would  likely  pay  somewhere  between 
$12,000  and  the  $22,000  per  mile  (persona!  communication  June  4, 1992). 

What  are  the  mainr  msi  rnnsiderations? 

•  Consultation-  Charges  for  consultants  vary  greatly,  with  costs  dependent  upon  detail,  distance, 
and  location.  Services  include  (a)  feasibility  studies,  (b)  route  surveys,  (c)  coordinate 
specifications,  (d)  right-of-way  checks,  (e)  and  route  designs  with  drawings.  In  a  route  design  the 
entire  distance  is  mapped.  In  a  city,  the  map  would  be  more  detailed  and  would  show  underground 
cables,  gas  lines,  and  electric  cables.  Also  included  in  some  consultation  fees  are  costs  for  staking 
the  route,  supervising  the  construction,  and  crew  scheduling  . 

Consulting  services  average  $5,000-$l 5,000.  At  engineering  companies,  consulting  costs  are  charged 
at  the  engineering  rate  of  $50  per  hour.  In  one  cost  study,  consulting  costs  range  from  $3  000-$3  500 
for  rural  areas  to  $7300  for  urban  areas. 

•  Construction /m tmals.  The  MWR  cost  study  identified  construction  and  material  costs  (Appendix 
D).  Labor  construction  costs  ranged  from  $0.97  per  foot  to  $1.55  per  foot  depending  upon  distance. 
Construction  costs  differ  between  rural  and  urban  loca-tions.  Urban  areas  require  more  field  work, 
engineer  work,  concrete  work,  and  easement  attainment.  The  costs  listed  in  Appendix  D  are  for 
suburban  locations  with  various  open  areas,  not  downtown  metropolitan  areas.  A  downtown  urban 
area  requires  additional  concrete  work  costing  $10  per  foot  or  more. 

Material  costs  are  estimated  to  be  $9,672  for  rural  constmction  and  $15,712  per  mile  for  urban 
construction.  Material  costs  include  fiber,  manholes,  warning  tape,  warning  signs,  and  splices.  In 
urban  areas,  duct  costs  are  also  added,  and  additional  splicing  and  manhole  costs  are  required. 

•  Easements-  If  construction  crosses  road-ways  or  railroads,  an  application  usually  must  be  filed  to 
obtain  an  easement  right-of-way.  Most  easements  are  obtained  as  public  right-of-ways  by  formal 
apphca-tion  to  the  city,  county,  or  state  entity.  If  a  public  easement  is  obtained,  a  license  is 
needed  to  place  the  cable  in  this  right-of-way.  Cities  often  charge  approximately  $1  per  foot  per 
year  for  easements.  Other  entities  typically  charge  a  percent  of  revenues  for  easements.  When 
crossing  private  land,  it  may  be  necessary  to  pay  for  easement  rights  to  the  private  individual. 

•  Terminal  equipment-  Digital  end  point  equipment  varies  in  price  according  to  speed  of 
transmission  and  capacity.  Digital  end  equipment  is  usually  one  unit  that  functions  as  the  codec, 
laser  trans-mitter,  optical  receiver,  and  multiplexor/  demultiplexer  (MUX/DEMUX).  Equip-ment 
cost  estimates  range  from  $25,000-$57,000,  including  installation.  Digital  terminal  equipment, 
connectors,  and  mstal-lation  of  this  equipment  costs  approximately  $70,000.  Analog  end  point 
equipment  costs  $4,000.  Costs  for  leasing  fiber  usually  include  the  costs  for  all  equipment  except  for 
the  device  that  converts  the  video  signals  generated  in  the  distance  education  classroom  into 
signals  that  can  be  transmitted  through  the  fiber.  This  device  costs  approximately  $6000  -  $8000. 

Microwave  Systems 

Microwave  signals  are  transmitted  through  the  air  electromagnetically.  Microwaves  can  carry  full 
motion  video.  The  placement  of  towers,  equipment  configurations,  federal  regulations,  and  frequency 
specifications  necessitate  techni-cal  assistance  and  custom-made  bids. 

For  a  short  distance,  point-to-point  system  between  two  schools,  microwave  would  be  an  appropriate 
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choice.  Microwave  systems  are  easy  to  install  especially  if  transmitters  are  placed  on  buildings. 

Microwave  signals  can  be  analog  or  digital.  Analog  is  less  expensive  since  equipment  is  not  needed  to 
create  digital  signals.  Digital  costs  are  decreasing,  but  many  organizations  continue  to  utilize  analog 
systems.  Most  companies  only  install  new  analog  microwave  systems  to  be  compatible  with  previously 
constructed  analog  systems.  When  constructing  an  entirely  new  system,  digital  microwave  is  generally 
used  because  it  provides  higher  quality.  Analog  is  affected  by  weather  and  atmospheric  conditions 
that  can  cause  fading,  which  produces  noise  and  creates  a  hiss  on  the  line. 

What  dpes  it  cost  to  install  a  microwave  system? 

The  average  cost  to  construct  a  microwave  system  covering  a  short  distance  is  $40,000.  Long-haul,  one- 
hop  systems  over  8-15  miles  can  cost  $150,000-$250,000.  The  use  of  repeater  towers  adds  expenses  for 
additional  electronics,  path  studies,  installation  costs,  and  tower  costs.  Each  repeater  tower  adds 
approximately  $90,000  in  equipment  costs  and  $20,0O0-$35,0O0  in  tower  construction  costs. 

What  are  major  qost  considerations? 

•  Consultation.  Services  include  (a)  path  profile  to  check  for  obstructions;  (b)  frequency  coordinates 
to  find  other  frequencies  that  might  interfere;  (c)  path  analysis  to  determine  how  much  power  is 
required;  (d)  completion  of  the  FCC  application;  and  (e)  tower  specifications  to  note  items  such  as 
ice  and  wind  load. 

Consulting  services  cost  $2,000-$4,000,  with  an  additional  $2,000  for  tower  specifications.  A  one- 
hop  path  profile/  coordination  usually  costs  about  $1300. 

•  Tower  construction.  Besides  distance,  tower  height  is  dependent  upon  obstructions,  elevation  above 
sea  level,  and  earth  curvature.  To  transmit  20  miles,  the  antenna  must  be  at  least  150-200  feet  on 
both  ends.  The  200-foot  tower  is  average  in  Iowa.  Towers  over  farm  land  can  be  fairly  short,  but 
any  trees,  buildings,  or  grain  elevators  in  the.  path  will  increase  the  height  requirement. 

A  20-foot  tower  is  the  minimum  height  for  a  transmitter.  In  order  to  clear  trees,  a  50-60-foot 
height  is  normally  required.  To  place  a  dish  on  a  building,  the  building  must  be  high  enough  to 
clear  trees  and  obstructions.  Antenna  extensions  attach  to  a  wall  and  extend  beyond  the  building 
roof.  A  20-foot  tower  in  Iowa  is  generally  a  building  or  side-mount  type  and  would  cost 
approximately  $5,000.  An  antenna  on  top  of  a  two-story  school  building  can  send  signals  only  a 
short  distance. 

A  100-foot  tower  costs  approximately  $20,000.  A  200-foot  tower  costs  approximately  $30,000  and  a 
300-foot  tower  costs  approximately  $35,000.   Construction  costs  are  higher  for  self-supporting 
towers,  but  less  land  is  required  since  no  guy  wires  are  needed.  A  200-foot,  self-supporting  tower 
costs  approximately  $200,000.  A  short,  light  weight  tower  for  a  short  distance  transmissions  may 
cost  as  little  as  $2,000-$3,00O.  A  heavier  tower  that  attaches  to  the  side  of  a  building  and  extends 
up  to  100  feet  wittrost  about  $5,000. 

•  Land,  Land  is  needed  for  the  tower  site.  A  200-foot  tower  requires  two  acres  of  land.  It  is 
sometimes  desirable  to  lease  land  if  possible.  A  land  lease  will  usually  cost  $500-$800  a  year  in 
rural  areas  and  as  high  as  $5,000-$6,000  per  year  in  urban  areas. 

•  Equipment  The  following  equipment  is  needed  at  both  ends  of  the  system-transmitters,  receivers, 
antennas,  hardware  connections.  This  equipment  costs  on  average  $45,000  at  each  end  for  long  haul 
systems  and  $11,000  per  end  for  short  haul  distances. 

•  Building,  It  is  often  necessary  to  construct  a  building  to  house  the  electronics  at  each  tower  site. 


16      COST  ANALYSIS 


Building  construction  is  approximately  $1300. 

Connections  tQ  the  school.  The  tower  should  be  located  within  a  mile  of  the  school.  If  towers  can 
not  be  located  near  the  school,  short  haul  systems  are  required  to  get  from  the  tower  to  the  school 
Coaxial  cable  used  to  connect  the  tower  to  the  school  costs  about  $5(000-$6(000  per  mile  and  an 
amplifier  is  needed  every  2,000  feet  ($500  each).  At  the  school,  the  cable  is  hooked  to  a  translator 
channel  box  for  television  reception,  or  to  a  VCR  connection. 

Installation  and  testing.  Before  buying  a  microwave  system,  it  is  advantageous  to  place  equipment 
on  trucks  and  transmit  from  the  point-to-point  locations  to  verify  that  there  is  no  interference. 
Installation  and  testing  of  equipment  and  systems  cost  up  to  $15,000-$30,000  per  tower  location. 

FCC  application-  A  completed  FCC  application  form  #402  is  required.  The  application  fee  is 
$155.  Multiple  hops  require  more  than  one  license.  Noncommercial  educational  broadcasts  are 
exempt  from  charges  (Code  of  Federal  Regulations,  1991,  §1.1112  (c)).  A  letter  to  the  FCC  would  be 
necessary  to  request  a  fee  exemption.  The  FCC  requires  that  frequency  coordinations  must  be 
completed  by  microwave  technical  experts,  at  a  cost  of  approximately  $1,000  to  $1,500. 

lOPg  and  Short  haul  systems.  The  maximum  distance  between  towers  is  about  30  miles.  Any 
distance  over  30  miles  would  require  repeater  towers  to  boost  and  retransmit  signals.  Distances 
closer  to  the  30  mile  limit  require  higher  towers,  more  powerful  transmitters,  and  larger  antennas 

S2f  ^ateI  toWer  needs  two  5615  of  «!u'Pment'  P°»"ted  each  direction  and  adds  approximately 
$100,000  to  the  cost  of  the  system,  plus  additional  installation  and  testing  costs.  Short  haul 
systems  can  be  purchased  for  as  little  as  $15,000  to  $20,000  per  site. 

Compressed  Video  Sy^TTlff 

As  used  here  compressed  refers  to  digital  video,  audio,  and  data  signals  that  are  processed  to  reduce 
the  amount  of  information  required  for  trans-mission.  This  process  reduces  the  bandwidth  requirement 
Compressed  signals  can  be  sent  using  any  transmission  method.  Compressed  video  systems  do  not 
transmit  full  motion  video,  but  do  allow  users  to  economically  transmit  signals  on  smaller  bandwidths. 

What  does  a  compressed  video  system  rp^? 

Equipment  Is  neededto  compress  signals  and  change  signals  from  analog  to  digital.  This  equipment  costs 
approximately  $36,000-$38,000  per  site.  1 

•  Consultation-  These  services  are  often  included  in  the  total  cost  of  vendor's  packages  Design 
coordinators  provide  input  about  the  equipment  needed  for  particular  classrooms. 

•  Equipment-  Transmission  equipment  con-sists  of  a  codec,  interface,  and  a  leased  transmission  line 
Codecs  averages*  ,000.  CSU/DSUs  cost  approximately  $1,700.  Transmission  line  lease  rates 
vary  from  $5,000-$20,000  a  year,  depending  on  the  vendor  and  the  distance. 

•  Installation-  Charges  for  installing  codec  equipment  are  often  included  in  the  codec's  purchase 
price,  but  some  vendors  charge  as  much  as  $1 300  for  this  service. 

•  Upgradps.  Most  vendors  now  have  codecs  that  can  be  upgraded  by  updating  their  software.  This  is 
an  essential  because  this  technology  is  changing  so  rapidly.  These  upgrades  cost  $8,000-$15,000. 

•  Transmission  line  lfasmatons-  When  leasing  services,  there  is  normally  a  one  time  connection 
charge  of  approximately  $1,400;  this  charge  is  not  related  to  distance.   Lease  rates  vary 
considerably.  The  longer  the  distance,  the  less  per  mile  per  month.  Year  lease  rates  range  from 


ERIC 


COST  ANALYSIS  17 


ERIC 


$5,000-520,000  for  distances  of  3  to  75  miles.  Most  schools  apply  for  and  receive  FCC-1  tariff  rates. 
If  the  origination  and  receive  sites  are  not  in  the  same  Local  Access  Transport  Area  (LATA), 
contacts  must  be  made  with  the  local  carrier  who  will  then  make  arragements  with  the  distant 
site's  carrier. 

It  is  possible  to  lease  terminal  equipment  (codecs).  The  list  cost  is  multiplied  by  a  cost  factor  to 
arrive  at  the  monthly  charge.  A  $36,000  codec  leases  for  $1,136  per  month,  or  $13,632  a  year,  based 
on  a  36-month  contract.  Shorter  contracts  cost  more  per  month.  For  a  12-month  lease,  the  monthly 
charge  averages  $2,800  (or  $33,600  per  year). 

Other  Results 

How  do  the  costs  vary  for  distances  from  ?  miles  to  75  miles? 

Transmission  using  fiber  and  microwave  are  directly  affected  by  distance.  Fiber  is  affected  the  most 
dramatically.  The  costs  per  mile  are  very  high  and  each  additional  mile  can  add  $12,000-$20,000  to 
the  cost  of  a  system. 

Microwave  costs  are  also  affected  by  distance.  The  longer  the  distance,  the  higher  the  tower,  and  the 
larger  the  antenna  dish  needed.  A  6-foot  dish  is  $1,500,  compared  to  a  12-foot  dish  which  costs 
approximately  $6,000.  Every  increment  of  20-30  miles  requires  a  new  repeater  tower  with  two  sets  of 
electronics.  Each  repeater  tower  can  add  $90,000-$1 20,000  for  equipment,  $20,000-$35,000  for  tower 
construc-tion,  and  approximately  $25,000  for  installa-tion  and  testing.  Path  studies  for  each 
additional  hop  cost  $1,000-$1,500. 

Compressed  video  systems  are  the  least  affected  by  distance.  Most  of  the  cost  of  a  system  is  in  the  codec 
equipment  at  the  end  points.  Compressed  video  systems  are  fairly  constant  in  cost. 

What  are  the  recurring  and  maintenance  costs? 

Transmission  equipment  is  usually  quite  reliable.  However,  service  and  maintenance  are  important 
considerations  for  these  systems.  Technology  does  not  always  work  and  environmental  circumstances 
may  necessitate  costly  repairs.  Maintenance  contracts  should  state  how  readily  the  vendor  is  expected 
to  provide  service. 

pifrer.  Fiber  equipment  is  reliable  and  needs  few  repairs.  Replacement  of  a  laser  may  cost  $2,000  to 
$4,000,  but  laser  failure  is  not  common.  Maintenance  contracts  for  end  point  equipment  are  about  one  to 
two  percent  of  the  purchase  price  per  month.  This  amount  averages  $3,300-$7,600  per  year. 

Microwave.  Circumstances  that  require  maintenance  on  microwave  systems  are  wind  damage,  ice  load, 
electrical  storms,  and  electronic  equipment  failure.  Maintenance  contracts  vary,  and  range  between  one 
to  ten  percent  of  the  system's  purchase  price. 

Compressed  video.  Maintenance  contracts  range  from  three  to  seven  percent  of  the  purchase  cost.  A 
recurring  expense  is  the  lease  cost  of  the  transmission  line. 

What  are  the  characteristics,  advantages,  and  disadvantages  of  the  three  trapsrpission  technologies? 

Eibfil.  Fiber  is  superior  in  signal  quality  and  bandwidth  capacities.  Fiber  is  unaffected  by  the 
environment  and  needs  little  maintenance.  Fiber  costs  per  channel  capacity  are  cost-effective.  The 
disadvantages  of  fiber  are  high  initial  cost  and  the  complexity  of  installation. 

Microwave.  Microwave  transmission  for  short  distances  is  cost-effective.  Short  distances  use 
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inexpensive  towers  and  relatively  low  cost  transmission  equipment. 

One  disadvantage  of  microwave  is  that  it  is  an  aging  technology,  and  the  majority  of  frequencies  in 
metropolitan  areas  have  been  assigned.  Distance  education  systems  require  a  great  deal  of  channel 
capacity.  In  the  last  five  years,  the  increasing  use  of  cellular  phones  has  decreased  the  number  of 
available  frequencies.  For  short  haul  microwave  systems  (which  utilize  higher  frequencies)  there  are 
more  frequencies  available. 

In  certain  areas,  microwave  may  not  be  feasible  because  of  the  terrain.  Microwaves  are  affected  by 
weather  conditions,  and  some  find  FCC  licensing  and  adherence  to  regulations  restrictive. 

Compressed  gidfi&  Compressed  video  is  easy  to  install  and  is  relatively  inexpensive  as  compared  to 
the  other  systems.  For  educational  purposes,  compressed  video  may  technically  not  be  the  best  but  it  is 
the  easiest  and  often  the  least  expensive  to  install.  Since  compressed  video  does  not  transmit  full 
motion,  images  appear  jerky  at  times.  However,  research  data  indicate  that  for  most  distance 
education  applications,  the  quality  is  quite  acceptable. 

Distance  Education  Classroom  Costs 

This  section  presents  the  costs  for  an  interactive  distance  education  classroom.  Technical  experts 
interviewed  about  classroom  design,  production  equipment  requirements,  types,  and  specification 

What  does  it  Cost  to  ennin  a  distant  pH.jcation  interactive  rlassrnnm? 

Equipment  for  the  classroom  designed  for  this  monograph  cost  $39,539  for  the  state-of-the-art 
cosTStimatw  $21'988     *  baSk  chSSr°°m-  Thtf  APP^i*6*      include  equipment  specifications  and 

C^njmllmg.  Private  design  consultants  charge  approximately  $5,000  to  develop  a  plan  for  a  complete 
1ST;,     ?       3  Resources  Center  at  Iowa  State  University  charges  $250  for  an  initial  visit  and 
$250  a  day  plus  expenses  for  additional  visits.  Installation  assistance  costs  are  $30  per  hour  per  person 
Consultation  costs  for  installing  a  complete  classroom  would  average  about  $4,000. 

RpomTreatTTient.  Most  classrooms  are  easily  adapted  for  use  as  distance  education  classrooms.  A  room  is 
adequate  if  there  is  sufficient  lighting  for  the  cameras,  no  extensive  glare  problems,  adequate  wirine, 
and  adequate  space.  Although  most  rooms  have  adequate  wiring  for  distance  education  equipment,  it  is 
necessary  to  determine  the  power  requirements  for  all  equipment.  Technicians  can  then  verify  that 
there  are  sufficent  outlets  and  amperage  levels.  General  rules  to  follow  include  (a)  unless  it  is  a 
special  circuit,  do  not  put  more  than  five  items  to  a  line,  and  (b)  isolate  computers  on  separate  lines. 

An  acoustically  treated  room  prevents  many  problems  associated  with  audio  systems.  Acoustic 
treatment  includes  fiber  ceiling  tiles,  carpet,  drapes,  or  fabric  covered  panels.  A  room  can  be 
acoustically  treated  for  $3,000-$4,000. 

Most  classroom  production  equipment  is  reliable  and  does  not  require  repair,  but  replacement  costs  for 
broken  microphones  or  cameras  may  be  needed. 

Classroom.  Design-  Examples  of  distance  education  classrooms  are  found  in  the  Appendix.  The  key  to 
the  design  is  the  equipment  list.  Costs  reflect  current  catalog  list  prices.  School  systems  can  expect  a  30 
percent  discount  from  vendors. 
i 

Equipment  PffyripHpn.'r.  The  size  of  the  representative  classroom  in  the  design  is  28  feet  by  30  feet  and 
will  accommodate  at  least  24  students.  Tables  are  24  inches  x  96  inches,  each  Lring  four  sidents.  Two 
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monitors  and  two  microphones  are  located  on  each  table.  The  instructor  desk  can  be  placed  upon  a  riser 
to  improve  visibility.- 

•  Cameras.  CCD  cameras  (charged  coupled  device)  use  a  "chip"  in  the  pick-up  device  rather  than  a 
conventional  tube.  The  chip  is  more  durable  and  requires  less  light.  Included  in  the  camera  list 
price  are  a  pan /tilt  controller,  a  zoom  lens,  and  AC  adapters.  The  pan/tilt  controller  and  zoom 
lens  can  be  eliminated  from  the  cameras  pointed  at  students,  but  are  necessary  on  the  instructor 
camera.  A  2-chip  camera  produces  a  higher  quality  signal  in  lower  light.  However,  a  1-chip 
camera  is  acceptable  for  most  applications.  Three-chip,  studio  quality  cameras  can  be  used,  but  are 
more  expensive. 

The  visual  presenter  (document  camera)  features  a  single  chip,  CCD  color  camera  with  manual 
focus  and  zoom  control.  An  overhead  camera  can  be  ceiling  mounted  as  a  substitute  for  the 
presenter.  However,  the  visual  presenter  is  easier  to  install  and  use. 

•  Monitors.  Twenty-five  inch  or  larger  color  monitors  should  be  used  at  the  origination  and  remote 
sites  to  display  program  output.  At  the  origination  site,  a  monitor  should  also  display  the  remote 
students.  Three  7  inch  color  monitors  are  used  at  the  teacher  station.  One  monitor  displays  the 
remote  site  students,  one  the  program  output,  and  one  continually  shows  the  document  camera's 
output  so  the  instructor  can  align  materials  before  sending  the  signal  from  this  device.  The  13  inch 
color  monitors  on  student  tables  allow  for  display  of  the  program  video.  Individual  student 
monitors  are  recommended  because  (a)  larger  projection  screens  lose  resolution  and  have  a  degraded 
image,  (b)  the  closer  the  distance,  the  better  the  visibility  and  legibility,  and  (c)  if  one  monitor  of 
the  system  malfunctions  others  are  available.  At  the  remote  site,  an  additional  monitor  is 
sometimes  added  in  a  teacher's  or  principal's  office  for  supervisory  purposes. 

Multiscan  monitors  automatically  accept  inputs  from  any  video  or  personal  computer  source.  If 
there  is  a  need  to  transmit  computer  data,  then  monitors  must  be  multiscan. 

•  Audio  systems.  Although  the  ideal  dis-tance  education  classroom  would  have  a  microphone  for 
each  person,  two  people  per  microphone  is  adequate.  The  best  microphones  for  the  distance 
education  classroom  are  cardioid.  Cardioid  microphones  pick  up  sounds  better  from  the  front  than 
the  back.  Cardioid  microphones  are  used  where  ambient  noise  should  be  suppressed. 

Student  microphones  should  be  low  profile,  wired,  and  voice  activated.  Low  profile  microphones 
require  minimal  installation.  The  teacher  microphone  should  be  a  wireless  or  wired  lapel-type. 

Push-to-talk  microphones  are  not  recom-mended.  They  require  an  action  on  the  part  of  the  student 
and  are  not  "user  friendly."  The  clicking  of  the  button  is  also  a  noise  in  the  system.  Frequent 
repairs  are  need  when  this  '      of  microphone  is  used. 

•  Media  controller.  A  switcher  lets  the  instructor  select  and  control  multiple  video  outputs,  and  a 
remote  control  manipulates  cameras,  in  a  state-of-the-art  classroom,  a  room  controller  functions  as 
a  switcher,  controls  the  local  and  remote  sites,  and  controls  camera  functions.  It  can  also  have 
diagnostic  functions. 

•  Rack  mount.  The  rack  mount  shelving  unit  fits  under  or  beside  the  desk.  Adapter  kits  netd  to  be 
added  to  shelve  individual  units  ($30-$150  each  unit). 

•  Video  distribution.  A  video  distribution  amplifier  is  needed  if  many  monitors  are  used.  This  unit 
divides  and  amplifies  video  signals. 
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SUMMARY 

This  study  determined  the  costs  for  creating  an  interactive,  two-way  distance  education  system  using 
fiber,  microwave,  and  compressed  video.  Transmission  costs  were  compared  for  various  distances.  The 
study  also  described  costs  for  equipping  and  installing  a  distance  education  classroom. 

Each  medium  was  found  to  be  unique.  When  selecting  a  medium,  some  compromises  may  be  required- 
budget  constraints  or  environmental  factors  may  force  trade-offs.  Significant  elements  to  consider  are 
the  content  of  material  communicated,  the  quality  of  transmission  desired,  the  cost,  and  the  ease  of 
system  use. 

Fiber  systems  were  found  to  be  more  complex  than  the  other  media.  Microwave  and  compressed  video 
systems  had  elements  that  were  comparable  from  system  to  system.  It  was  most  difficult  to  estimate 
costs  for  fiber  systems.  There  are  more  cost  variables  to  consider,  and  these  variables  are  not  consistent 
between  applications. 

If  full  morion  video  is  necessary,  microwave  systems  are  viable  alternatives  for  schools.  Microwave 
systems  are  cost-effective  and  easy  to  install  if  the  transmission  distance  is  less  than  10  miles 
Microwave  is  also  the  least  expensive  option  for  short  distances.  Leasing  microwave  towers  can 
decrease  costs  for  two-hop  systems,  but  leasing  still  requires  a  large  investment. 

Microwave  transmissions  can  be  affected  by  weather  conditions.  Adverse  weather  can  also  inflict 
damage  on  microwave  equipment.  Microwave  frequencies  are  not  always  available,  and  geographic 
topography  may  limit  microwave  feasibility.  ear 

Compressed  video  systems  offer  the  quickest  and  easiest  solution  for  creating  an  ITV  network  between 
two  schools.  A  compressed  video  system  can  be  installed  with  little  need  for  extensive  construction 
Most  educators  say  they  prefer  a  full  motion  system,  but  research  indicates  that  compressed  video  is 
effective  and  satisfactory  and  that  the  picture  quality  of  the  compressed  video  technology  is  constantly 
improving.  Most  viewers  do  not  object  to  small  sacrifices  of  color,  motion,  or  resolution.  Successful  users 
indicate  that  content  takes  precedence  over  image  quality. 


Concerning  the  three  transmission  systems,  it  was  found  that: 


Fiber  was  found  to  be  most  expensive  and  fiber  systems  were  more  complex  than  other  media. 
Microwave  for  distances  under  10  miles  was  a  cost-effective  way  to  connect  two  schools 
The  major  costs  were  (a)  system  consulting  and  design,  (b)  materials,  construction,  and  installation 
(c)  terminal  end  point  equipment,  and  (d)  maintenance  contracts. 

Distance  significantly  affected  the  costs  of  fiber  and  microwave.  Both  became  cost-prohibitive 
ways  to  connect  two  sites  as  distances  increased. 

Compressed  video  system  costs  were  the  least  affected  by  distance.  Most  of  the  costs  for  this 
technology  were  for  compression/end-point  equipment. 

Most  experts  recommended  using  digital  technology,  which  offers  superior  quality 
Fiber  can  be  cost-effectively  installed  by  schools  short  distances  from  each  other. 
Urban  fiber  construction  costs  were  greater  than  rural  fiber  construction  costs. 
Microwave  frequencies  are  not  available  in  some  cities. 

Leasing  a  building  or  tower  for  use  as  a  repeater  tower  between  two  schools  rather  than  constructing 

a  tower  decreased  expenses.  6 

Leasing  a  repeater  station  required  a  large  capital  investment. 

There  was  variation  in  microwave  repeater  lease  rates  according  to  location. 

There  was  variation  in  land  lease  rates  according  to  the  location  of  microwave  towers 

Towers  should  be  built  as  close  to  schools  as  possible. 

Installing  and  testing  microwave  systems  was  expensive. 

Long  haul  microwave  systems  were  not  cost-effective  unless  a  repeater  structure  was  leased 
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inexpensively,  and  dishes  were  put  on  buildings. 

•  One  hop  microwave  costs  (1  -  30  miles)  increased  as  the  distance  increascd-higher  towers  and 
more  powerful  transmitters  were  needed. 

•  Microwave  systems  were  the  most  susceptible  to  malfunctions,  because  they  were  most  affected  by 
weather. 

•  Telephone  companies  charged  more  than  utility  companies  for  fiber  leases. 

•  Metropolitan  areas,  generally,  had  access  to  less  expensive  fiber  leases  than  rural  areas. 

•  Cost  per  mile  for  lease  rates  tor  DS1  &  DS3  decreased  as  distance  increased. 

•  Lease  rates  for  DS3  were  3  to  8  times  the  monthly  lease  rate  of  DS1. 

Conclusions  about  the  interactive  distance  education  classroom  were  that: 

•  Usually  classrooms  do  not  have  to  be  completely  remodeled  to  be  distance  education  classrooms. 

•  Basic  equipment  costs  ranged  from  $25,000  -  $30,000. 

•  Equipment  was  easily  added  or  eliminated  to  fit  budgets  and  needs. 

•  Packages  of  complete  systems  cost  30  to  70  percent  more  than  the  purchase  of  individual  items. 

•  Maintenance  is  not  a  significant  cost  variable  for  classroom  equipment. 

•  Transmission  of  computer  generated  signals  significantly  increased  costs. 

•  There  was  variation  in  vendors  equipment  packages  in  relation  to  capabilities,  included  options, 
and  prices. 


Conclusions 

If  full  motion  video  is  essential  for  interactive  distance  education,  microwave  is  the  most  viable 
solution  for  shorter  distances.  Even  though  microwave  systems  are  affected  by  weather,  they  still  are 
effective  for  linking  two  schools.  Digital  fiber  is  the  best  choice  as  far  as  quality  and  capacity,  but 
high  costs  tend  to  negate  this  advantage.  Fiber  is  the  most  complex  to  install;  this  complexity 
contributes  to  higher  costs.  High  costs  make  fiber  too  expensive  for  two  school  systems  to  install 
without  some  outside  financial  assistance.  Leasing  fiber  is  appropriate  if  fiber  is  in  place. 

If  compressed  video  is  deemed  a  satisfactory  transmission  method,  overall  it  is  the  least  expensive  to 
install  and  is  a  viable  solution,  no  matter  the  distance.  Compressed  video  is  a  cost  appropriate  solution 
for  longer  distances.  Both  fiber  and  microwave  systems  are  somewhat  cost  prohibitive  for  longer 
distances. 

The  standard  classroom  b  easily  adapted  for  use  as  a  distance  education  classroom  without  significant 
remodeling  costs.  A  distance  education  classroom  can  be  equipped  for  less  than  $30,000. 

Two-way,  interactive  television  systems  are  expensive,  complex  and  time-consuming  to  design, 
construct,  and  maintain.  Once  installed,  however,  they  are  neither  difficult  nor  complex  to  use.  With 
knowledgeable  advice  and  sufficient  planning,  schools  can  have  successful  distance  education  programs. 
If  the  purpose  of  choosing  an  interactive  system  is  to  reach  students  and  provide  instruction,  all  three 
approaches  can  attain-that  goal  on  a  fairly  equal  basis.  The  success  of  any  delivery  system,  of  course, 
depends  upon  the  quality  and  usefulness  of  the  content  delivered  and  received,  rather  than  upon  the 
choice  of  equipment. 
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APPENDIX  A-  Considerations  For  Distance  Education  Systems 


CONSIDERATIONS  FOR  SETTING  UP  A  DISTANCE  EDUCATION  SYSTEM 


Fiber  System 


2. 


Feasibility  study.  This  is  a  planning  stage.  Specific  options  to  consider  are: 
Service  needs 

#  Economic  and  technical  aspects 

•  Procuring  easements  from  the  railway  or  state,  city  or  county 
^     Contacting  individuals  currently  utilizing  fiber  systems 

Leasing  from  individual  telephone  companies,  utility  companies,  cable 
companies,  and  large  businesses 

•  Researching  all  possible  partnerships  and  funding  sources 
Growth  requirements 

SYSTEM  design.  This  includes  determining  specifications,  routing  distances,  and 
estimating  costs.  Specific  decisions  and  concerns  include: 

#  Type  of  system  -  analog  or  digital 
Transmission  distance 

Type  of  fiber  -  single  mode  or  multimode,  step  index  or  graded  index 
Type  of  optical  equipment  -  LED  or  ILD;  PIN  or  APD 
Modulation  type  and  code  format;  wavelength 
Path  coordinates 
Detail  map  of  route 
Easements  for  roadways  and  railroads 

If  leasing,  negotiate  contract  with  conditions  on  rates,  maintenance,  service 
priority,  and  time  periods 

Purchase  and  install.  This  phase  involves  actually  putting  the  fiber  into  the 

SpedflcS  t0fl8ther  With  t8rminal  equipment'  and  testin9  the  system. 

Method  of  installation 
Staking  the  route 
Splices  or  connectors 
Installing  the  fiber 

Terminal  equipment  -  cost  and  technical  support 
Initial  check  of  the  system  to  see  if  it  works 
Maintenance  agreement/fees 
Warranty 

Maintenance  support 

Compatibility  of  end  point  equipment  with  classroom  equipment 


Microwave  System 

1 '       c^sider^  STUDY'  ThiS  iS  th8  planning  sta9e-  The  following  are  options  to 
Service  needs 

Economical  and  technical  aspects 
Funding  and  financial  planning 

Contacting  individuals  currently  utilizing  microwave  systems 
Availability  of  frequencies  or  leasing  facilities 
Land  topography  and  distance 
Growth  requirements 

Availability  of  suitable  sites  for  repeater  stations 
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2.  System  design.  This  includes  determining  specifications,  locating  points,  and 
estimating  costs.  Specific  decisions  and  concerns  include: 

#  Type  of  system  -  analog  or  digital 

#  The  distance  and  number  of  repeaters  needed 

#  Selection  of  sites  -  must  be  line-of-sight  with  no  obstructions  as  trees,  or 
buildings 

#  Visual  check  of  lease  possibilities  and  obstructions 

#  Frequency  band  study  to  find  other  frequencies  that  might  interfere 

#  Setting  frequency 

#  Path  profile  to  determine  tower  height  and  check  for  obstructions  -  use 
topographical  map  to  determine  elevations  and  plot  points 

#  Tower  specifications,  e.g.,  wind  and  ice  load 

#  Path  calculations  to  determine  equipment  parameters/  configurations  to  meet 
performance  requirements,  e.g.,  antenna  size,  transmitter  power  output, 
receiver  noise  figure,  required  bandwidth 

#  Path  survey  to  provide  information  vital  to  installation;  review  above 
calculations  and  specifications 

#  If  leasing,  negotiate  contract  with  rates,  maintenance,  service  priority,  and  tine 
periods 

3.  Purchase  and  install.  This  phase  involves  the  actual  construction  of  the  tower, 
installing  transmission  and  terminal  equipment,  and  testing  equipment.  Specific 
decisions  and  concerns  include: 

#  Acquiring  land-lease  or  buy 

#  Availability  of  power  and  access  road 

#  Getting  FAA  clearance 

#  Filing  FCC  application  which  includes  an  interference  analysis  and  system 
design 

#  FCC  tower  construction  application  if  over  200  feet 

#  Constructing  the  tower  -  consider  time  schedule 

#  Installing  the  transmission  equipment;  build  protective  building 

#  Beam  alignment,  equipment  lineup  and  checkout 

#  Connections  from  tower  to  school 

#  Hooking  up  cable  in  classroom 

#  System  warranty 

#  Maintenance  agreement/fees 

#  Maintenance  support 


Compressed  Video  System 

1 .       FEASIBILITY  STUDY.  This  is  the  planning  stage.  The  following  are  options  to 
consider: 

•  ServiceTTeeds 

•  Economic  and  technical  aspects 

•  Possibilities  for  leasing  the  cable 

*  Cost-sharing  between  two  sites 

#  Deciding  if  compressed  video  has  the  quality  needed  for  the  determined 
educational  use 

*  Contacting  individuals  currently  utilizing  compressed  video  systems 
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?nctTcEManHSIGN"  J"?  inClud6S  determinin9  equipment  specifications,  estimating 
costs,  and  negotiating  contracts.  The  following  are  specific  decisions  and 
concerns*. 

Transmission  equipment  (codec)  -  purchase  or  lease 

•  Transmission  rate  -  1 12  kb/s,  384  kb/s,  1 .544  Mb/s 

•  Dial  up  or  dedicated  line 

•  P  x  64  standards  for  interoperability/proprietary  standards 
Full  or  fractional  T  lease 

Purchase  and  install. 

•  iLee3t!ir9  T1  H"eS  "  ne9°.tiate  rates,  maintenance,  service  priority,  time  periods 
Installing  equipment  -  time  schedule,  costs 

•  Technical  support  for  installation  equipment 

•  Purchasing  own  classroom  equipment  or  vendor  video  package 
Compatibility  of  all  equipment  in  system 

•  Upgrading  capabilities 

•  Technical  hardware  and  software  service  support 
Warranty  on  equipment 

•  Maintenance  agreement/fees 
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APPENDIX  B  -  Cost  Estimates  For  Fiber  Systems 
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APPENDIX  C  -  Cost  Estimates  For  Microwave  Systi 
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DIGITAL  RADIO  SYSTEM 
BASIC  SYSTEM  DUPLEX  (Non-Protected) 


Equipment 


SHORT  HAUL  3  MILES 
Unit  Price 


Transmitter/Receiver  $10,300 
w/  battery  unit 
Antenna  (included) 


Point 
A 

1 


Point 
B 

1 


Point 
C 


Cable 

Audio  Subca.rier 
Tower 
Path  Study 
Installation/Test 


600 
930 
3.000 
1,000 
1,500 


System 
Total 

$20,600 


1.200 
1,860 
6,000 
2,000 

3, OOP 
$34,660 


TERMS  USED  ON  THE  FOLLOWING  PRICE  LISTS  ARE  DEFINED  BELOW: 
Duplex  refers  to  two-way  communication. 
Audio  subcarrier  refers  to  equipment  to  put  audio  on  video  channel. 
Waveguide  refers  to  antenna  feed  from  antenna  to  transmitter. 
Dehydrator  refers  to  unit  that  keeps  the  waveguide  moisture  free. 
Charger  system  refers  to  the  battery  power  unit. 

High  power  (TWfTand  the  auto  power  control  refers  to  a  power  amplifier  unit. 

Non-Protected  refers  to  equipment  that  does  NOT  supply  automatic  redundancy  to  cover 
system  failures. 
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DIGITAL  RADIO  SYSTEM 
BASIC  SYSTEM  DUPLEX  (Non-Protected) 


SHORT  HAUL    10  MILES 
Equipment  Unit  Price 

Transmitter/Receiver  $10,300 


Point 
A 


Point 
B 


Point 
C 


Antenna  4  feet 
Ant  Cable 
Tower 
Path  Study 
Installation/Test 


1.500 
600 
5,000 
1.000 
2.000 


System 
Total 

$20,600 

3,000 
1.200 
10.000 
2.000 
4.000 
$40,800 


•  his  system  includes  a  heavier  extension  tower  on  the 
building  roof  that  extends  to  at  least  100  feet.  The  antenna  size  is  increased  over  the  3  mile 
distance  antenna  adding  an  additional  $1,500  to  the  cost. 


:rlc 
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DIGITAL  RADIO  SYSTEM  (1  DS3  CAPACITY) 
BASIC  SYSTEM  DUPLEX  (Non-Protected) 


SINGLE  HOP    20  MILES 


Equipment 

Transmitter/Receiver 
Antenna  8  foot 
Waveguide  (per  foot) 
Ant/WG  Hardware 
Dehydrator 

Charger  System  12  Amps 
Batteries  96  Amp  Hour 
Tower  200  foot 
Path  Study 
installation/Test 


Unit  Price 

$32,720 
3,200 
13 
2.500 
758 
2.095 
937 
20,000 
1.500 
16,000 


Point 
A 

1 

1 

250 

1 

1 

1 

1 

1 


Point 
B 

1 

1 

250 

1 

1 

1 

1 

1 


Point 
C 


System 
Total 

$65,440 

6.400 

6,500 

5.000 

1,512 

4.190 

1.874 

40.000 

3,000 

32.000 

$165,916 


ERIC 
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DIGITAL  RADIO  SYSTEM  (1  DS3  CAPACITY) 
BASIC  SYSTEM  DUPLEX  (Non-Prot«cttd) 


SINGLE  HOP    30  MILES 


Eauiament 

Unit  Dri 

Trans  mitter/Receivpr 

Hiah  Power  fTWT^ 

Auto  Power  Control 

1 

Antenna  8  foot 

3.200 

Waveguide  (per  foot) 

13 

Ant/WG  Hardware 

2.554 

Dehydrator 

850 

Charger  System  25  Amps 

3.700 

Batteries  200  Amp  Hour 

1.500 

Tower  300  foot 

35,000 

Path  study 

1.50C 

installation/Test 

15,000 

Point 
A 


20 


Po 


nt 


320 


Point 


Systtm 
Total 

$65,440 
8,000 
3.100 
6,400 
8,320 
5.108 
1,700 
7,400 
3,000 
70,000 
3.000 
32,000 
$213,468 


ERIC 
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DIGITAL  RADIO  SYSTEM  (1  DS3  CAPACITY) 
BASIC  SYSTEM  DUPLEX  (Non-Protected) 

TWO  HOP   SO  MILES 
20  mile  hop  &  30  milt  hop 


Equipment 

Unit  Price 

Point 

Point 

Point 

System 

A 

B 

C 

Total 

Transmitter/Receiver 

$32,720 

1 

2 

1 

$130,800 

High  Power  (TWT) 

4,000 

1 

1 

8,000 

Auto  Power  Control  (APC) 

1,550 

1 

1 

3,100 

Antenna  6  foot 

1,529 

1 

1 

3,058 

Antenna  8  foot 

3,200 

1 

1 

6,400 

Waveguide  (WG)  per  foot 

13' 

320 

640 

320 

16,000 

Ant/WG  Hardware 

2,554 

1 

2 

1 

10,216 

Dehydrator 

756 

1 

1 

1,512 

Dehydrator 

850 

1 

850 

Charger  System  12  Amps 

2,095 

1 

1 

4,190 

Chargor  System  25  Amps 

3,700 

1 

3,700 

Batteries  96  Amp  Hour 

937 

1 

1 

1,874 

Batteries  200  Amp  Hour 

1,500 

1 

1,500 

Tower  300  foot 

35,000 

1 

1 

1 

105,000 

Path  study 

1,500 

4,500 

Installation/Test 

16,000 

1 

1 

32,000 

Installation/Test 

24,000 

1 

24.000 

$356,700 


DIGITAL  RADIO  SYSTEM  (1  DS3  CAPACITY) 
BASIC  SYSTEM  DUPLEX  (Non-Prottct«d) 


POUR 
25 

HOP  75  MILES 
mile  hops 

Equipment 
Transmitter/Receiver 

Unit  Price 
$32,720 

Point 
A 

1 

Point 
B 

2 

Point 
c 

2 

Point 
O 

1 

System 
Total 

$196,320 

High  Power  (TWT) 

4,000 

1 

1 

1 

1 

16,000 

Auto  Power  Control 

1,550 

1 

1 

1 

1 

6,200 

Antenna  0  foot 

1,529 

1 

2 

2 

1 

9,174 

Waveguide  (WG)per  foot 

13 

280 

560 

560 

280 

21,840 

Ant/WG  Hardware 

2t700 

1 

2 

2 

1 

16,200 

Dehydrator 

850 

1 

1 

1 

1 

3,400 

Charger  System  12  Amps 

2,095 

1 

1 

4,190 

Charger  System  25  Amps 

3,700 

1 

1 

7,400 

Batteries  96  Amp  Hour 

937 

1 

1 

1,874 

Batteries  200  Amp  Hour 

1,500 

1 

1 

3,000 

Tower  250  feet 

30,000 

1 

1 

1 

1 

120,000 

Path  Study 

1.500 

7,500 

Installation/Test 

16.000 

1 

1 

32,000, 

Installation/Test 

24.000 

1 

1 

4fl,000 

$493,098 
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APPENDIX  D  -  Cost  Estimates  For  Compressed  Video  Systems 
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(REV.  April  20.  1992) 


DISTANCE  EDUCATION  PROJECT 


==.==_!MIMARY  EQUIPMENT  SPECIFICATIONS  AND  COST  P^TlMATCg 

MODEL 


DESCRIPTION  MAKE 
I .    ORIGINATION  SITE  INSTRUCTOR  CONSOLE 


==.OG.  SPEAKER 

COMPUTER 

VISUAL  PRESENTER 

SLIDE  TO  VIDEO  TRANSFER 

VHS  PLAYER 

WS  RECORDER 

=£MOTE  SITE  MONITOR 

'-AVALIERE  MIC 

BACKUP  WIRED  MIC 

=REVtEW/PROGRAM  MONITOR 

FURNITURE/RACKS 

PAX  MACHINE 


ANCHOR 

APPLE  MAC  II  SI 

ELMO 

ELMO 

PANASONIC 

PANASONIC 

PANASONIC 

SHURE 

SHURE 

SONY 

WINSTED 


AN-1000X 

W/NU-VISTA  BD 

EV-308/LU-3C8 

TRV-35G 

AG-1730 

AG-1730 

PVM-8220  IN  DUAL  CONFIG 
AMS  23 
AMS  24 

PVM-8220  IN  DUAL  CONFIG 


ORIGINATION  SITE  DISTRIBUTION  EQUIPMENT 


CODEC  4  CONTROLLER 
■  AUDIO/VIDEO  DIST.AMP 
AUDIO/VIDEO  SWITCHER 
=.=M.CCNT.  CAMERAS 
AJDIO  MIXER/ALC  SYSTEM 


SHURE 

:;.  ORIGINATION  SITE  STUDENT  STATIONS 

SPEAKERS  FOR  SONY  MCN  ANCHOR 

-JRNITURE  .  .  LUXOR 

MICROPHONES  SHURE 

VIEWING  MONITORS  SONY 

FURNITURE  TABLES 

•V.  REMOTE  SITE  DISTRIBUTION  EQUIPMENT 


EXTRON  OR  TECH 
PANASONIC  WV-D51C0 


AMS  4000 


AN-10C1 
ATW-56 
AMS  22 
PVM-2S30 


CCOEC  &  CONTROLLER 
AUDIO/VIDEO  DIST.AMP 
AUDIO/VIDEO  SWITCHER 
-■Brf.CONT.  CAMERAS 
AUDIO  MIXER/ALC  SYSTEM 
"JRNITURE/RACKS 


E2 

EXTRON  OR  TECH 
PANASONIC  WV-D5100 
SHURE  AMS  4000 

WINSTED 


PRELIMINARY  EQUIPMENT  REQUIREMENTS  FOR  THE 
COMPRESSED  VIDEO  CLASSROOM 

DATA  PRESENTED  TO  THE  CLASSROOM  STUDY  COMMITTEE 


UNIT 

sue 

300.00 

300.00 

5000.00 

5000.00 

2600.00 

2600.00 

2300.00 

2300.00 

500.00 

500.00 

500.00 

.500.00 

1200.00 

1200.00 

200.00 

200.00 

225.00 

225.00 

1200.00 

1200.00 

1000.00 

1000.00 

800.00 

800.00 

1 

35000.00 

35000.00 

1 

400.00 

400,00 

1 

1500,00 

1500.00 

2 

2800.00 

5600.00 

1 

1800,00 

1800.00 

6 

200.00 

1200.00 

3 

300.00 

900.00 

8 

200.00 

1600,00 

3 

1500.00 

4500.00 

8 

200.00 

1600.00 

1 

35000.00 

35000.00 

1 

400.00 

400.00 

1 

1500.00 

1500.00 

2 

2800.00 

5600.00 

1 

1800.00 

1800.00 

1 

1000.00 

1000.00 

(REV.  April  20.  1992) 


DESCRIPTION  MAKE 

MODEL 

QTY 

UNIT 

SUB 

V.    REMOTE  SITE  MONITOR  CONSOLE 

FAX  MACHINE 

? 

] 

800.00 

800.00 

VISUAL  PRESENTER 

ELMO 

EV-308/LU-308 

2500.00 

2500.00 

VHS  RECORDER 

PANASONIC 

AG-1730 

] 

500.00 

500.00 

BACKUP  WIRED  MIC 

SHURE 

AMS  28 

200.00 

200.00 

PREVIEW/PROGRAM  MONITOR 

SONY 

PVM-8220  IN  DUAL  CONFIG 

1 

1200.00 

1200.00 

REMOTE  SITE  MONITOR 

SONY 

PVM-8220  IN  DUAL  CONFIG 

1200.00 

1200.00 

FURNITURE/RACKS 

WINSTED 

1 

1000.00 

1000.00 

VI.  REMOTE  SITE  STUDENT  STATIONS 

FURNITURE 

? 

8 

200.00 

1600.00 

SPEAKERS  FOR  SONY  MON 

ANCHOR 

AN-1001 

4 

200.00 

800.00 

VIDEO  CARTS 

LUXOR 

ATW-56 

2 

300.00 

600.00 

MICROPHONES 

SHURE 

AMS  22 

8 

200.00 

1600.00 

VIEWING  MONITORS 

SONY 

PVM-2530 

2 

1500.00 

3000.00 

EQUIPMENT  TOTAL 

VII  INSTALLATION  COSTS 

CABLING/CONDUIT 
LABOR 

VIII.  LINE  LEASING  * 


BELDEN&PANDi; 


$  130225.00 


2000.00 


2000.00 


IX.  SYSTEM  MAINTENANCE/SUPERVISION  SUPPORT  • 
MOVING  AND  SETUP  COSTS  (S250/DAY) 
TROUBLE  CALLS  FROM  ISU  TO  SITE 

TOTALS 


EQUIPMENT 
INSTALLATION 
UNE  LEASING 
SYSTEM  MAINTENANCE 

TOTAL 


$  130225.00 
2000.00 


•ANNUAL  ONGOING  COSTS 


Lk  O  h 


APPENDIX  E  -  Classroom  Designs  and  Equipment  Lists 


Distance  Education  Classroom  Equipment  List 
Origination  Site 


Qamejas  Quan 
A     Student  camera  (Panasonic  WV-F70) 

B  Student  camera 
C    Teacher  camera 

Mounting  brackets  camera  (Pan  WV131P) 
I)    Visual  presenter  (Elmo  EV-308) 

llghtbar  (Elmo  LU-308) 
Monltora 

E     Monitor,  outgoing  25"(Sony  PVM2530) 

V     Monitor,  remote  site  25" 

G     Teacher  monitor,  remote  site  25" 

H     Teacher  monitor,  outgoing  25" 

I     Monitor,  individual  student  12" 

(NEC  PM1271A) 
J     Monitor,  teacher  T  (Panasonic  BT-S701N) 

Mounting  brackets  monitor  (celling) 

(Bretford  Mfg  TVM1) 

K     Speakers  (Anchor  AN  1001) 

K2  Teacher  speaker  (Anchor  AnIOOOX) 

Brackets  (Anchor  SB730) 
L     Microphone,  student  (Shure  AMS22) 
M    Microphone,  teacher/wireless 

(Telex  FMR-25TD) 
Taanhar  station 

N     Audio/Video  control  (Crestron) 

O     Teacher  control  unit 

Rack  mount  shelving  (Wlnstead) 
Audio  mixer  (Shure  AMS8000) 
Video  distribution  (Panasonic  WJ300B) 
Power  strip  (Wlnstead  98700) 
Light/dimmer  control  (AMXM320) 

p     VCR  play/record  (Panasonic  AQ1250) 

Q     Facsimile  (HP200) 


Supplies,  cabling,  wiring,  connectors 


Cost/Unit 

Total  Cost 

1 

4295 

4295 

1 

4295 

4295 

1 

4295 

4295 

3 

52 

156 

1 

3100 

3100 

1 

300 

300 

2 

1590 

3180 

2 

1590 

3180 

1 

1590 

1590 

1 

1590 

1590 

12 

525 

6300 

3 

600 

1800 

6 

221 

1326 

4 

1 

4 

12 

1 


163 
324 
43 
210 
600 


652 
324 
172 
2520 
600 


1  6000 

6000 

1  500 

500 

1  2800 

2800 

2  290 

580 

1  69 

69 

1  675 

675 

2  350 

700 

1  1200 

1200 

EQUIPMENT  TOTAL 

$52199 

•  30% 

$36539 

3000 

TOTAL 

$39539 

ERIC 
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f  t  491 


Quantity       Cost/Unit        Total  Cost 


Options  for  SHftfitltUtipn  or  anhannflmgnt 

3  chip  camera  (Panasonic  WVF250) 

Tripod  for  camera  (ITE-T30) 

Large  screen  monitor  35"  (Sony  PVM5310) 

Multiscan  monitors  13"  (Sony  GVM-1300) 
20"  (Sony  GVM-2000) 
35"  (Mitsubishi  AM-3501R) 

b&w  teacher  display  monitors  4"  (PVM  411) 

Metal  cart  for  monitor  (Winstead  R3353) 

Lavalier  microphone  (Shure  AMS28) 

Room  control  panel  (PenPal  USWest) 

Slide  to  video  converter  (Elmo  TRV-35G) 

Film  to  video  converter  (Elmo  TRV-16G) 

Echo  canceller  (Shure) 

Time  Base  corrector  (NOVA  700) 

Computer  interface  (Micro  386  board) 

NTSC  Genlock  card 

Video  Scan  Converter  PS  2 

Video  Scan  Converter  MAC 

Table  96"  x  24"  Barron 

Table  18"  x  60"  Barron 

Table  60"  x  24"  Barron 

Porcelain  board/metal  board  72  x  48  $6.20  sq.ft. 


Acoustic  treatment 
Carpet 
Wall  panels 
Drapes 


14.00  -16.00  sq. 
5.80  aq.  ft 
6.25  sq.  ft 


yd 


7800 
295 
2625 
1495 
2300 
6900 
1540 
210 
235 
7000 
3215 
4290 
1640 
2290 
2200 
2000 
12995 
9995 
606 
483 
518 
149 


Riser 


Materials 
Materials 
Labor 


8'  x  16' 
8'  x  24' 


Consultation:  $1,250-65,000 
Installation:  $2,500-$5.000 


160 
240 
100 


ERIC 
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Basic  Classroom 
Distance  Education  Classroom  Equipment  List 
Origination  Site 


Cameras  Quantity 

oosuunn 

1  Well  wU9t 

A 

Student  camera  w/o  pan,  tilt,  zoom 

1 

1  /  tU 

1  79ft 

C 

Teacher  camera  (Panasonic  WVD5100) 

1 

Mounting  brackets  camera  (Pan  WV131P) 

2 

1  a  a 

Teacher  camera  control 

1 

4fln 

D 

Visual  presenter  (Elmo  EV-308) 

1 

0 1  A  fx 
3  1  OU 

01  Aft 

O  1  uu 

iightbar  (Elmo-LU308) 

1 

on  ft 

Oftft 

Monitors 

E 

Monitor,  outgoing  25'  (Sony  PVM2530) 

1 

4  E  A  A 

1CQO 

1  D9U 

F 

Monitor,  remote  site  25" 

1 

1  090 

1  CQft 

G 

Teacher  monitor,  Remote  site  25" 

1 

1  590 

1  can 
1  D  v  U 

H 

Teacher  monitor,  Outgoing  25" 

1 

1  C  A  A 

1590 

1  con 

Mounting  brackets  monitor  (celling) 

4 

221 

(Bretford  Mt  TVM1) 

J 

Monitor,  teacher  T  (Panasonic  BT-S701N) 

3 

600 

1800 

AqdiQ 

K 

Speakers  (Anchor  AN1001) 

4 

163 

ceo 
052 

K2 

Teacher  speaker  (Anchor  AN1000X) 

1 

324 

324 

Brackets  (Anchor  SB730) 

4 

43 

172 

L 

Microphone,  student  (AMS22) 

12 

210 

M 

Microphone,  teacher/iavalier  (ShureAMS28)  1 

235 

235 

Teacher  station 

N 

Audio/Video  control  (Extron  Model  8) 

1 

1495 

1405 

0 

Teacher  control  unit 

Rack  mount  shelf  (Winstead) 

1 

500 

500 

Video  dlst.(Panasonic  WJ300B) 

1 

290 

290 

Audio  mixer  (Shu  re  AMS8000) 

1 

2800 

2800 

Power  strip  (Winstead  98700) 

1 

69 

69 

P 

VCR  play/record  (Panasonic  AG1250) 

2 

350 

700 

Q 

Facsimile  (HP200) 

1 

1200 

1200 

EQUIPMENT  TOTAL 

S285SS 

-  30% 

$19988 

Supplies,  cabling,  wiring,  connectors 

$2,000 

$21988 

:RIC 
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Title: 

The  Effectiveness  of  Instructional  Orienting  Activities 
in  Ckunputer-Based  Instruction 


Author: 
Richard  F.  Kenny 
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Introduction 


The  low  cost  and  accessibility  of  the  microcomputer  has  placed  the  power  and 
flexibility  of  computing  into  the  hands  of  the  individual  and  opened  a  world  of  instructional 
possibilities.  Among  these  are  the  capabilities  offered  by  computer-based  instruction  (CB1) 
and  of  interactive  multimedia  such  as  computer-based  interactive  video  (CBIV)  in  particular. 
While  the  use  of  CBI  has  been  extolled  for  its  capability  to  individualize  instruction,  interactive 
multimedia  combines  the  power  of  the  computer  to  support  student  interaction  with  the 
richness  of  the  various  audio  and  visual  media. 

According  to  Jonassen  (1984),  the  major  advantage  of  CBIV  (and  CBI  in  general)  is  that 
it  is  both  adaptive  and  interactive.  He  defines  adaptive  as  "the  ability  to  adapt  or  adjust  the 
presentation  sequence,  mode  or  sign  type  to  meet  a  variety  of  instructional  requirements..." 
and  interactive  as  "that  the  program  engages  the  learner  to  participate  in  a  variety  of  ways  that 
utilize  learner  responses..."  (p.21).  For  Hannafin  (1989),  interactivity  is  more  detailed, 
encompassing  various  instructional  capabilities  such  as  confirmation  of  response,  learner 
control  of  pacing  and  lesson  sequencing,  inquiry  (glossaries  and  libraries)  and  elaboration, 
techniques  which  allow  learners  to  combine  known  with  to-be-learned  lesson  information. 

CBI,  and  especially  interactive  multimedia,  while  providing  great  flexibility,  is  not 
without  problems.  One  is  the  potential  for  learner  disorientation,  the  loss  of  one's  sense  of 
location  or  of  the  structure  of  the  material.  Navigation  is  the  most  commonly  identified  user 
problem  in  hypermedia  (Jonassen,  1989,  Kinzie  k  Berdel,  1990,  Rezabek  k  Ragan,  1989). 
Learners  can  easily  become  lost  and  frustrated  and  may  give  up  without  acquiring  any 
information  from  the  program. 

Another  potential,  and  related,  problem  is  cognitive  overload.  Jonassen  (1989)  also 
notes  that  the  exponentially  greater  number  of  learning  options  available  to  learners  places 
increased  cognitive  demands  upon  learners  that  they  are  often  unable  to  fulfil.  Tripp  and 
Roby  (1990)  claim  that  disorientation  leads  to  the  expenditure  of  more  mental  effort  to 
maintain  a  sense  of  orientation  in  the  program  which  in  turn  reduces  the  mental  resources 
available  for  learning. 

A  major  challenge  for  teachers  and  instructional  designers  is  to  learn  how  to  make 
effective  use  of  the  capabilities  of  such  interactive  learning  systems  to  assist  people  to  learn 
while  avoiding  the  inherent  problems.  To  guide  research  on  the  design  of  instructional 
prescriptions  for  CBI,  the  ROPES+  meta-model  (Hooper  k  Hannafin,  1988,  Hannafin  k 
Rieber,  1989)  was  proposed.  ROPES+  refers  to  Retrieval,  Orientation,  Presentation,  Encoding, 
and  Sequencing,  'V  the  influence  of  contextual  factors.  The  "O"  in  ROPES+,  then,  refers  to 
any  form  of  "Orienting"  activity  which  acts  as  "a  mediator  through  which  new  information  is 
presented  to  the  learner"  (Hannafin  k  Hughes,  1986,  pg.  239).  Included  in  this  category  are 
attention-gaining  techniques,  lesson  objectives,  pre-questions  and  advance  organizers,  which 
the  authors  consider  to  be  "a  variation  of  cognitive  orienting  ability"  (Hannafin,  1987,  p.48). 

It  is  this  category  of  orienting  activities  that  may  suggest  methods  for  alleviating  the 
problem  of  disorientation  and  cognitive  overload  in  CBI.  Tripp  k  Roby  (1990),  for  instance, 
suggest  that  it  has  been  the  advance  organizer  which  has  traditionally  been  used  as  a  device  to 
orient  students  to  content.  However,  other  related  instructional  organizers  might  also  be 
useful.  Among  these  are  the  structured  overview  graphic  organizer  (Barron,  1969)  and  the 
pictorial  graphic  organizer  (Hawk,  McLeod  k  Jonassen,  1985),  both  derivatives  of  the  advance 
organizer. 

The  purpose  of  this  paper  is  to  review  the  research  literature  pertaining  to  the  use  of 
instructional  organizers  and  to  provide  a  comparative  analysis  of  their  effectiveness  with  CBI. 
The  review  will  first  consider  what  evidence  there  is  that  any  of  these  techniques  have  an 
effect  on  learning  or  retention.  Second,  it  will  examine  the  relevant  research  on  the  use  of 
such  orienting  techniques  with  CBI.  Third,  the  paper  will  compare  two  cognitive  theories, 
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Ausubel's  (1963)  Subsumption  Theory  and  Wittrock's  (1974)  GeneraHve  Learning  Hypothesis, 
to  try  to  discern  which  most  strongly  predicts  the  effectiveness  of  these  techniques.  A  third 
theory,  Schema  Theory  (Anderson,  1977,  Rumelhart  it  Ortony,  1977,  Jonassen,  1989),  and  a 
related  Instructional  orienting  technique,  the  hypermap  (Jonassen,  1989,  Reynolds  St 
Dansereau,  1990,  Reynolds  et  al.,  1991),  are  suggested  as  an  alternative  for  further 
investigation. 


Instructional  Organizers 

One  of  the  earliest  forms  of  instructional  organizer  was  the  advance  organizer  first 
proposed  by  David  Ausubel  (1960, 1963).  The  advance  organizer  is  meant  to  facilitate  the 
retention  of  meaningful  verbal  information.  It  is  introduced  in  advance  of  the  learning 
material  itself  and  presented  at  a  higher  level  of  abstraction,  generality  and  inclusiveness 
(Ausubel,  1963).  Since  its  main  function  is  to  bridge  the  gap  between  the  learner's  cognitive 
structure  and  the  material-to-be-learned,  the  advance  organizer  must  be  stated  in  terms 
familiar  to  the  learner. 

The  graphic  organizer  was  first  advanced  as  a  "structured  overview  by  Barron  (1969) 
as  a  modification  of  the  advance  organizer  and  later  renamed  (e.g.  Barron  It  Stone,  1974).  It  Is 
a  tree  diagram  which  introduces  the  new  vocabulary  to  be  used  in  the  matertal-to-be-leamed 
and  uses  the  spatial  characteristics  of  diagrams  to  indicate  the  relationships  and  distances 
between  key  terms  (Hawk,  McLeod  and  Jonassen,  1985).  While  introduced  as  an  advance 
organizer,  the  structured  overview  graphic  organizer  is  unlike  the  former  because  it  is  written 
at  the  same  level  as  the  to-be-learned  material  and  uses  lines,  arrows  and  spatial  arrangement 
to  depict  text  structure  and  relationships  among  key  vocabulary  (Alverman,  1981). 

Hawk,  McLeod  and  Jonassen  (1985)  further  developed  Barron's  modification.  Their 
form  of  organizer  is  a  more  pictorial,  visual,  or  graphic  presentation  than  the  two  previous 
organizers.  Pictorial  graphic  organizers  take  one  of  two  forms:  participatory  organizers,  In 
which  students  participate  in  the  completion  of  the  organizer,  and  final  form  organizers,  in 
which  they  do  not. 


Research  on  Advance  Organizers 

Ausubel's  studies  . 

Ausubel  and  his  associates  (Ausubel,  1960,  Ausubel  St  Fitzgerald,  1961, 1962,  Ausubel 
St  Youssef,  1963),  provided  some  of  the  most-cited  research  supporting  the  effectiveness  of  the 
technique.  Ausubel  (1960)  tested  the  learning  of  undergraduates  from  a  2500  word  passage  on 
metallurgy  and  produced  statistically  significant  results  in  favour  of  the  group  receiving  an 
expository  advance  organizer.  Ausubel  St  Fitzgerald  (1961)  compared  the  effects  of  an 
expository  advance  organizer  and  a  comparative  organizer  on  learning  from  a  2500  word 
passage  on  Buddhism.  The  comparative  organizer  group  significantly  outperformed  the 
expository  group  on  a  posttest  given  after  three  days,  but  there  was  no  significant  difference 
between  the  expository  and  control  (descriptive  passage)  groups.  A  posttest  given  after  10 
days  indicate*  that  both  organizer  groups  retained  significantly  more  of  the  material  to  be 
learned  than  the  control  group. 

Ausubel  St  Fitzgerald  (1962)  also  conducted  a  second  study  to  compare  the  effects  of 
an  expository  advance  organizer  on  learning  from  two  sequential  passages  on  endocrinology. 
No  significant  main  effect  was  shown  for  either  passage.  However,  a  significant  main  effect 
was  demonstrated  for  subjects  in  the  lower  third  subgroup  of  a  test  of  verbal  ability  as 
predicted  by  advance  organizer  theory.  Finally,  Ausubel  St  Yousaef  (1963)  compared  the  effects 
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of  a  comparative  advance  organizer  on  learning  material  from  a  passage  on  Zen  Buddhism  to 
a  control  group.  They  reported  a  significant  main  effect  for  the  organizer  treatment  when  both 
verbal  ability  and  knowledge  of  Christianity  (to  which  Buddhism  was  compared  in  the 
organizer)  were  controlled. 

These  studies  appear  to  have  demonstrated  that  advance  organizers  do  facilitate 
learning.  A  recent  detailed  analysis  of  these  four  studies  (McEneany,  1990),  however,  has 
revealed  a  number  of  problems  and  calls  into  question  these  results.  McEneany  claims  no 
consistent  evidence  across  the  four  studies  in  support  of  advance  organizers  nor  for  predicted 
interactions  with  verbal  ability.  He  suggests  that  "a  sound  operational  definition  of  an 
advance  organizer  eludes  even  Ausubel  himself  (p.  95),  a  claim  previously  advanced  by  other 
writers  (e.g.  Hartley  &  Davies,  1976,  Lawton  &  Wanska,  1977,  Macdonald-Ross,  1978,  Clark  & 
Bean,  1982).  McEneany  (1990)  was  not  the  first  to  dispute  the  effectiveness  of  advance 
organizers.  Hartley  &  Davies  (1976)  reviewed  the  technique  as  a  part  of  a  broader  overview  of 
pre-instructional  strategies  and,  in  spite  of  conflicting  evidence,  concluded  that  advance 
organizers  did  facilitate  both  learning  and  retention.  They  also  claimed  thai  a  major  problem 
lies  in  the  design  and  writing  of  advance  organizers  because  there  were  no  procedures  nor 
operationally  defined  steps  for  generating  them.  Ausubel,  however,  has  disputed  this  on 
several  occasions  (e.g.  Ausubel,  1978,  Ausubel,  Novak  &  Hanesian,  1978),  claiming  that 
sufficient  guidelines  can  be  found  in  his  articles  and  books. 

Barnes  &  Clawson  (1975)  were  less  generous.  They  rated  32  studies  using  a  "voting 
technique"  based  on  whether  the  results  were  statistically  significant  or  not.  Non-significant 
results  prevailed  20  to  12  leading  the  investigators  to  a  negative  conclusion.  They  also 
differentiated  among  the  studies  according  to  length  of  study,  ability  and  subject  type.  In  each 
case,  the  count  favoured  non-significance.  The  Barnes  &  Clawson  (1975)  review,  however,  was 
itself  strongly  criticized  on  methodological  grounds  (Ausubel,  1978,  Lawton  &  Wanska,  1977 
Mayer,  1979a). 

The  Advance  organizer  and  assimilation  encodine  theory 

Mayer  (1979a)  reinterpreted  Ausubel's  subsumption  theory  in  terms  of  his  own 
assimilation  encoding  theory  and  reported  a  series  of  nine  studies  supporting  his  contention. 
This  theory  predicts  that  the  organizer  will  facilitate  both  the  transfer  of  anchoring  knowledge 
to  working  memory  and  its  active  integration  with  the  received  information.  It  also  predicts 
that  the  advance  organizer  may  have  no  effect  if  the  content  and  instructional  procedure 
already  contains  the  needed  prerequisite  concepts,  if  the  content  and  instructional  procedure 
are  sufficiently  well-structured  to  elicit  the  prerequisite  concepts  from  the  learner,  or  if  the 
organizer  does  not  encourage  the  learner  to  actively  integrate  the  new  information.  Further,  if 
the  learner  already  possesses  a  rich  set  of  relevant  past  experiences  and  knowledge  and  has 
developed  a  strategy  for  using  it,  the  advance  organizer  would  not  be  effective  (for  example  a 
high  ability  learner). 

Thus,  Mayer  stipulates  the  following  characteristics  for  constructing  advance 
organizers: 

1.  Short  set  of  verbal  or  visual  information. 

2.  Presented  prior  to  learning  a  larger  body  of  to-be-learned  information. 

3.  Containing  no  specific  content  from  the  to-be-learned  information. 

4.  Providing  a  means  of  generating  the  logical  relationships  among  the  elements 
in  the  to-be-learned  information. 

5.  Influencing  the  learner's  encoding  process 
(Mayer,  1979a,  p.  382). 

Mayer  (1979b)  also  reviewed  advance  organizer  literature.  Using  only  published 
studies  which  contained  either  an  advance  organizer  group  and  a  control  group  or  a  post 
organizer  group,  Mayer  rated  27  studies  according  to  three  questions: 
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1.  Is  the  material  unfamiliar,  technical  or  lacking  a  basic  assimilative  context? 

2.  Is  the  advance  organizer  likely  to  serve  as  an  assimilative  context? 

3.  Does  the  advance  organizer  group  perform  better  than  the  control  group  on  a 
test? 

He  listed  the  results  as  percentages  and  claimed  statistical  significance  for  only  three  studies. 
He  concluded  that,  when  used,  there  was  usually  a  small  but  consistent  advantage  for  the 
advance  organizer  group.  He  also  claimed  that  advance  organizers  more  strongly  aided 
performance  when  material  was  poorly  integrated,  that  they  more  strongly  aided 
inexperienced  learners  and  that  they  aided  transfer  more  than  specific  retention  of  details. 


Meta-analvses  of  Advance  Organizer  Research 

The  summary  reviews  discussed  previously  have  used  a  variety  of  techniques:  the 
traditional  literature  summary  (Hartley  &  Davies,  1976),  an  either/or  voting  technique  (Barnes 
&  Clawson,  1975),  and  a  question-based  voting  technique  (Mayer,  1979b).  Such  reviews  have 
been  strongly  criticized  as  overly  subjective  (e.g.  Wolf,  1986).  Two  later  reviews  of  the  advance 
organizer  research  literature  ,  however,  use  meta-analysis,  a  technique  which  permits 
quantitative  reviews  and  syntheses  of  the  research  issues  (Wolf,  1986).  One  frequently  cited 
meta-analytic  technique  is  Glass'  (e.g.  Glass,  McGaw  &  Smith,  1981)  effect  size  statistic  (E.S.) 
which  allows  the  comparison  of  studies  which  vary  in  design,  sample  selection  and  setting  in 
order  to  form  conclusions. 

The  first  meta-analysis  (Luiten,  Ames  &  Ackerson,  1980),  examined  135  studies 
yielding  110  E.S.  (Table  1)  for  learning  (posttest  within  24  hours  of  the  treatment)  and  50  E.S. 
for  retention  (posttest  24  hours  and  after).  They  calculated  a  mean  E.S.  of  0.21  for  learning, 
indicating  that  the  average  subject  receiving  the  advance  organizer  treatment  would  perform 
better  than  58%  of  the  control  group  individuals.  The  mean  E.S.  for  retention  was  0.26.  Their 
conclusion  was  that  advance  organizers  had  a  small,  facilitative  effect  upon  learning  as  well  as 
retention.  The  effect  on  learning,  however,  runs  contrary  to  Mayer's  predictions  based  on 
assimilation  encoding  theory  as  well  as  Ausubel's  subsumption  theory.  Luiten  et  al.  (1980)  also 
found  advance  organizers  effective  for  all  ability  levels  but  especially  for  high  ability  learners 
(a  mean  E.S.  of  0.23)  which  also  contradicts  both  Ausubel  and  Mayer. 

The  second  meta-analysis  (Table  1),  by  Stone  (1984),  more  closely  followed  Mayer's 
model.  Stone  included  only  studies  with  a  control  group  or  a  post  organizer  and  those  in 
which  a  posttest  was  administered  one  week  or  later  after  the  treatment.  She  listed  112 
different  E.S.  from  29  studies.  Stone  reported  an  overall  mean  effect  size  of  0,66  which 
indicates  that  advance  organizers  facilitate  the  long  term  retention  of  new,  unfamiliar  material. 
However,  she  also  compared  true  advance  organizers,  those  which  acted  as  subsumers 
(presented  at  a  more  general  or  abstract  level),  to  those  which  were  at  the  same  level  as  the 
material-to-be-learned.  The  mean  E.S.  for  the  subsuming  organizers  was  0.75  while  the  mean 
for  organizers  taken  from  the  material  was  0.71.  As  well,  Stone  (1984)  found  no  special 
facilitation  for  low  ability  learners.  While  generally  supportive,  these  results  contradict  two 
main  assumptions  of  both  subsumption  theory  and  assimilation  encoding  theory. 


Page  4 


499 


Table  1 

A  Comparison  of  Effect  Sizes  for  the  Luiten  et  al,  and  Stone 
Meta-analyses 


Variable 

Luiten  et  al. 

Stone 

learning 

0.21 

retention 

0.26 

0.66 

low  ability 

0.13 

0.26 

medium  ability 

0.08 

0.64 

high  ability 

0.23 

0.34 

A  Review  of  More  Recent  Advance  Organizer  Research 

Has  the  research  published  since  these  reviews  provided  any  further  evidence  to 
demonstrate  the  effectiveness  of  ad\  a  nee  organizers?  A  comparison  of  several  more  recent 
studies  (1984  -  1992)  of  advance  organizers  reflect  the  same  inconsistent  results  (Table  2') 
One  study  (Corkhill  et  al.,  1988)  consisted  of  six  experiments  to  investigate  retrieval  context  set 
theory.  This  theory  holds  that  re-reading  advance  organizers  before  the  posttest  will  aid 
retrieval  of  the  information  from  long-term  into  working  memory. 

Of  the  six,  three  experiments  used  advance  organizers  without  some  additional  activity 
such  as  paraphrasing.  The  mean  effect  sizes  for  these  experiments  are  3.75  for  the  cue 
condition  (re-reading  the  organizer  before  the  test  of  learning)  and  2.24  for  the  no  cue 
condition.  While  it  can  be  argued  that  the  cue  condition  might  have  added  a  practice  effect  to 
the  presumed  subsumption  function  of  the  advance  organizer,  the  no  cue  condition  (reported 
m  Table  2),  represents  the  use  of  advance  organizers  as  specified  by  Ausubel  and  Mayer 
Furthermore,  the  measure  of  learning  in  all  three  experiments  tested  retention.  An  average 
effect  size  of  2.24,  therefore,  is  strong  evidence  for  the  facilitating  effect  of  this  instructional 
technique. 

°'her  rec€nt  studies,  however,  produce  much  less  convincing  evidence.  Another 
study  by  Corkhill  (Corkhill,  Bruning  &  Glover,  1988)  compared  the  effects  of  concrete  and 
abstract  advance  organizers  on  students'  recall  of  prose.  The  concrete  organizer  was 
hypothesized  to  function  as  a  comparative  advance  organizer,  while  the  abstract  organizer  was 
expected  to  function  as  an  expository  advance  organizer.  The  abstract  organizer  treatments 
produced  a  mean  effect  size  of  -0.62  while  the  concrete  organizer  treatments  had  a  mean  effect 
size  of  2.25.  These  results  are  strongly  conflicting  since  they  provide  support  for  comparative 
but  not  expository  advance  organizers,  while  both  should  facilitate  learning  and  retention 

A  study  by  Lenz,  Alley  &  Schumaker  (1986)  investigated  the  effects  of  a  regular 
teacher's  delivery  of  an  advance  organizer  prior  to  each  lesson  on  Learning  Disabled  (LD) 
students'  retention  and  expression  of  information  from  a  given  lesson.   Student  learning  was 
assessed  by  an  after-class  interview  recording  the  number  of  statements  made  by  the  student 
related  to  the  lesson  in  which  the  organizer  was  used.  Results  indicated  improvement  both 
aHer  teacher  training  and  again  after  student  training  on  taking  notes  from  the  organizers, 
rne  first  improvement  can  be  attributed  to  the  use  of  the  advance  organizer  per  se  and 


Note:  Those  studies  concerned  with  the  use  of  advance  and  graphic  organizers  in 
CBI  (e.g.  Kenny  1992,  Tripp  &  Roby,  1990,  1991,  Carnes  et  al.(  1987)  are  included  in  Tables  2 
and  4,  but  will  be  discussed  in  a  later  section. 
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conflicts  with  Mayer's  theory  which  indicates  that  it  should  not  be  effective  for  learning.  The 
student  training  result  could  be  ascribed  as  much  to  the  generative  activity  of  note-taking  as  to 
the  advance  organizer. 

Kloster  &  Winne  (1989)  conducted  a  study  with  227  eighth  grade  mathematics  students 
who  were  randomly  assigned  to  one  of  four  treatment  groups:  (1)  expository  advance 
organizer,  (2)  comparative  advance  organizer,  (3)  outline  and  (4)  unrelated  passage  (control). 
A  mean  E.S.  of  -0.18  was  obtained  for  the  expository  advance  organizer  and  -0.15  for  the 
comparative  advance  organizer  indicating  a  slightly  negative  effect  for  advance  organizers. 
Studies  by  Gilles  (1984)  with  surgical  nursing  content  (ES.  =  033  for  retention)  and  Doyle 
(1986)  with  college  mathematics  (ES.  =  0.74  for  learning  and  E.S  =  1.03  for  retenion)  indicated 
expository  advance  organizers  affected  both  learning  and  retention  with  stronger  support  for 
the  latter,  thus  supporting  assimilation  encoding  theory. 

A  study  (Tajika,  Taniguchi,  Yamamoto  k  Mayer,  1988)  using  pictorial  advance 
organizers  with  fifth  grade  Japanese  mathematics  produced  similar,  and  stronger,  results. 
Mayer  (1979a,  p.  382)  indicated  that  an  advance  organizer  could  be  a  "short  set  of  verbal  or 
visual  [emphasis  added]  information".  Students  were  randomly  assigned  to  one  of  four 
groups:  (1)  an  integrated  pictorial  advance  organizer,  which  presented  two  geometric  figures 
divided  into  component  parts  in  an  organized  manner,  (2)  a  fragmented  pictorial  advance 
organizer,  which  presented  the  same  shapes  but  in  a  disorganized  way,  (3)  a  group  which 
received  extra  reading  time  and  (4)  a  control  group  which  merely  read  the  passage.  They 
studied  the  organizers  immediately  before  reading  a  550  word  passage  about  an  imaginary 
land  emphasizing  geometric  shapes.  Students  took  a  free  recall  (learning)  test  after  reading 
the  passage  and  a  delayed  recall  test  one  week  later.  Effect  sizes  were  highest  for  the 
retention  test  as  predicted  by  Ausubel  and  Mayer. 

Finally,  four  studies  (Carnes,  Lindbeck  k  Griffin,  1987,  Tripp  k  Roby,  1990,  1991, 
Kenny,  1992)  used  advance  organizers  with  computer-based  instruction  and  achieved  mixed 
results.  These  are  discussed  in  the  section,  Instructional  Organizers  with  CBI". 

Overall,  based  on  the  reviews  and  recent  studies  discussed  above,  the  research 
evidence  concerning  any  facilitative  effect  of  advance  organizers  upon  learning  and  retention 
is  quite  confusing.  Much  of  the  evidence  appears  positive,  and  yet,  it  is  clearly  quite  variable. 
Effect  sizes  for  the  more  recent  studies  range  from  -1*02  to  2.04  for  measures  of  learning  and 
from  -0.18  to  4.08  for  tests  of  retention.  It  may  be  that  this  range  represents  the  distribution  of 
study  means  around  the  true  effect  size  mean  for  this  instructional  technique.  The  variability 
would  reflect  differences  in  experimental  design,  subject  selection  and  methodology.  Given 
the  frequent  discussion  of  the  difficulty  in  constructing  an  advance  organizer  (e.g.  Hartley  & 
Davies,  1976,  McEneany,  1990),  one  likely  source  of  error  is  treatment  fidelity.  Much  of  this 
variability  may  reflect  the  lack  of  clarity  of  what  an  advance  organizer  is. 
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Table  2 


A  Comparison  of  Effect  Sirgs  for  Recent  Advance  Organizer  Studies 


Study 


Learning  Retention 


Corkhill  et  al.  (1988) 
Expt.  3  (after  1  week) 
Expt.  4  (after  24  hours) 
Expt.  5  (after  2  weeks) 

Corkhill,  Binning  &  Glover  (1988) 
Concrete  Organizer  -  Expt.l 
Concrete  Organizer  -  Expt.2 
Abstract  Organizer  -  Expt.l 
Abstract  Organizer  -  Expt.l 

Lenz,  Alley  &  Schumaker  1986 
After  teacher  training 
After  student  training) 

Kloster  &  Winne  (1989) 
Comparative  Organizer 
Expository  Organizer 

Gilles  (1984) 

Dovle  (1986) 

Tripp  &  Robv  (1990) 

Tripp  &  Robv  (1991) 

Carries.  Lindbeck  &  Griffin  (1987) 

Tatika,  Taniguchi.  Yamamoto  &  Mavpr 
(1988)   

Fragmented  Pictorial 

Integrated  Pictorial 

Kenny  (1992) 
Adv.  Org.  >  Partic.  Graph  Org. 
Adv.  Org.  >  Final  Form  Graph  Org. 


1.56 
2.93 
1.02 
0.21 


1.03 
2.93 


Mean 


0.015 

0.74 

1.25 

0.33 

0.49 


0.078 
2.04 


0.45 
1.17 

0.76 


1.96 
2.85 
1.91 


•0.15 
0.18 

0.33 

1.03 


0.14 


1.49 
4.08 


0.95 
0.45 

1.16 


If  advance  organizers  do  affect  learning,  how  do  they  achieve  this  effect?  The 

!EeorVSXPSrUPKaf '  nUbrmptS"  *"*  ,%3)  and  for  Nation  encoding 

theory  (Mayer  1979a)  calls  their  ability  to  predict  into  question.  Corkhill,  Bnining  and  Glover 

J"?.  £"2!  e*Planati°™         suggest  that,  rather  than  to  necessarily 
increase  discnminab.hty  between  prior  learning  and  new  material,  concrete  (comparative) 
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advance  organizers  furnish  ideational  anchorage  in  terms  already  familiar  to  the  learners,  that 
is,  to  provide  meaning  by  association  with  existing  schema.  Further,  they  may  assist  the 
learner  to  visualize  the  content  of  the  organizer  more  readily.  These  authors  also  stress  the 
importance  of  ensuring  encoding  of  the  organizer  using  techniques  such  as  paraphrasing, 
generative  techniques  to  be  discussed  below. 

In  summary,  the  jury  still  appears  to  be  out  on  the  advance  organizer  and  the  theory 
on  which  it  is  based.  What,  then,  of  the  other  variations  on  the  advance  organizer  concept? 
The  next  section  extends  this  discussion  to  include  two  of  its  progeny:  the  structured  overview 
form  of  advance  organizer  and  the  pictorial  graphic  organizer. 


Research  on  the  Use  of  Graphic  Organizers 


The  Structured  Overview  Form  of  Graphic  Organizer 

As  discussed  above,  the  graphic  organizer  was  first  presented  as  a  structured  overview 
(Barron,  1969)  and  meant  as  a  variation  on  the  advance  organizer.  Since  it  is  not  (e.g. 
Alverman,  1981),  the  question  arises  whether  there  is  evidence  that  this  form  of  organizer 
affects  learning  or  retention.  A  meta-analysis  by  Moore  k  Readance  (1984)  reported  an 
average  effect  size  of  0.22.  They  also  noted  an  average  effect  size  of  0,57  for  graphic  post 
organizers  constructed  by  the  instructor  with  the  class  or  by  the  student  alone,  They 
concluded  that  the  structured  overview  form  of  graphic  organizer  does  have  an  effect, 
especially  for  university  students,  that  vocabulary  learning  is  most  positively  affected  and  that 
post-organizers  benefit  learners  more  than  advance  organizers. 

More  recent  studies  appear  to  support  these  conclusions  (Table  4).  Alvermann  (1981) 
found  that  partially  complete  advance  graphic  organizers  had  an  effect  on  ninth  grade 
students'  comprehension  and  retention  of  text.  The  results  indicated  the  strongest  effect  for 
the  less  well  organized  text  as  predicted  by  assimilation  encoding  theory.  Two  studies 
(Boothby  k  Alvermann,  1984,  Alvermann  k  Boothby,  1986)  found  the  graphic  organizer  to  be 
effective  as  a  strategy  for  facilitating  fourth  graders  comprehension  and  retention  of  social 
studies  text.  While  the  second  (Alvermann  k  Boothby,  1986)  study  did  not  test  for  retention, 
it  did  indicate  that  graphic  organizers  had  an  effect  on  a  transfer  of  learning  task  -  again 
predicted  by  Mayer's  theory. 

Bean  et  al.  (1983)  reported  a  study  in  which  tenth  grade  world  history  students  were 
divided  into  three  groups;  those  taught  to  construct  summaries  and  post  graphic  organizers, 
those  taught  to  construct  post  graphic  organizers  only,  and  those  taught  to  build  outlines  only. 
Results  indicated  a  small  effect  for  combined  organizer  and  summary  training  group  (0,16)  but 
a  negative  effect  for  the  organizer  only  group  (-0.11).  Since  a  true  control  group  was  not  used 
in  this  study,  effect  sizes  were  calculated  using  the  outlining  group  as  a  control. 
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Table  3 

A  Comparison  of  Effect  Sizes  for  Recent  Structured  Overview  Graphic  Organize  ft..^fff 


Study 

Learning 

Retention 

Alvermann  (1981) 
Descriptive  Passage 
Comparative  Passage 

1.26 
0.06 

1.76 
0.41 

Boothbv  &  Alvermann  (1984) 

0.98 

0.99 

Passage  1 
Passage  2 
End  of  chapter  test 

-0.34 
0.45 
0.41 

Bean  et  al.  (1986) 

Organizer  and  Summary  Training 

Organizer  Only 

0.16 
-0.11 

Carr  and  Mazur-Stewart  (1988) 

0.89 

1.23 

Mean 

0.42 

1.10 

And  last,  Carr  k  Mazur-Stewart  (1988)  found  thai  a  vocabulary  overview  guide  (a 
multi-page  booklet)  which  included  a  graphic  organizer  was  significantly  superior  to  a 
traditional  form  of  instruction  in  improving  the  vocabulary  comprehension  and  retention  of 
college  students.  While  the  results  are  neither  extensive  nor  consistent,  taken  overall,  these 
studies  do  indicate  that  the  structured  overview  form  of  graphic  organizer  -  especially  the  do 
organizer  type  -  does  affect  learning  and  retention. 

The  Pictorial  Graphic  Oreanizer 

A  review  of  the  pictorial  graphic  organizer  research  is  reported  in  Table  4.  Two 
experiments  by  Jonassen  and  Hawk  (1984)  tested  the  use  of  teacher-constructed  participatory 
pictorial  graphic  organizers  in  regular  classrooms  (grade  eight  history,  grade  twelve  English 
literature).  The  results  indicated  a  stronger  effect  for  learning  than  for  retention.  Two  studies 
(Hawk,  McLeod  &  Jeane,  1981,  Hawk  k  Jeanne,  1983,  cited  in  Hawk,  McLeod,  k  Jonassen, 
1985)  reported  statistically  significant  results  in  favour  of  participatory  pictorial  graphic 
organizers.  However,  insufficient  data  was  available  to  calculate  effect  sizes.  A  more  recent 
study  by  Hawk  (1986)  also  found  this  technique  to  be  effective  in  facilitating  retention  for 
above  average  students  studying  life  science  in  the  sixth  and  seventh  grades. 

Darch,  Camine  k  Kameenui  (1986)  compared  several  techniques  for  informing  sixth 
grade  reading  students  of  content  information,  including  the  cooperative  (group)  and 
individual  completion  of  participatory  pictorial  graphic  organizers,  to  a  more  traditional 
directed  reading  approach.  The  graphic  organizers,  especially  for  the  cooperative  learning 
approach,  were  found  to  be  more  effective  in  facilitating  retention  than  the  traditional 
approach.  Learning  was  not  tested. 

Alvermann  (1988)  investigated  the  effects  of  a  final  form  graphic  organizer  (the  filled 
in  version)  designed  to  induce  tenth  grade  social  studies  students  to  look  back  in  their  texts 
for  missed  information.  The  organizer  facilitated  learning  for  self-perceived  low  ability 
students  (those  whose  ability  as  measured  by  a  standardized  achievement  test  matched  their 
own  perception  of  their  ability)  compared  to  a  control  group  of  self-perceived  low  ability 
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students  who  merely  read  the  passage  The  organizer  appeared  to  interfere  with  the  learning 
of  self-perceived  high  ability  students  compared  to  the  equivalent  control  group.  For  this 
study,  the  organizer  was  designed  as  a  road  map  to  guide  students  back  to  sections  in  the  text 
in  order  to  answer  posttest  questions. 

Table  4 

A  Comparison  of  Effect  Sizes  for  Pictorial  Graphic  Organizer  Studies 


Study  Learning  Retention 
fonassen  &  Hawk  (1984) 

Experiment  1  117  067 

Experiment  2  1-82  077 

Hawk  (1986)  —  064 

Parch,  Camine  &  Kameenui  (1986) 
Cooperative  Graphic  Organizer 

Individual  Graphic  Organizer  —  1-59 

K  ~  0.72 

Alvermann  (1988) 

Self-perceived  High  Ability  -  0.64 

Self-perceived  Low  Ability  3.94 

Kennv.  Crabowski.  Middlemiss,  059  -  0.07 
ft  y^n  NiHite-Kennv  (1991) 

Kennv  (1992) 

Participatory  Graph.  Org.  >  Adv.  Org.  -  0.45  -  0.95 

Final  Form  Graph.  Org.  >  Adv.  Org.  1.17  0.45 

Mean  109  0.48 


Finally,  studies  by  Kenny,  Grabowski,  Middlemiss  k  Van  Neste-Kenny  (1991)  and 
Kenny  (1992)  used  pictorial  graphic  organizers  with  CBIV  and  found  evidence  to  support  the 
technique.  Again,  these  are  discussed  in  the  section  on  organizers  in  CBI. 

In  summary,  the  evidence  reported  above,  while  once  again  inconsistent,  is  generally 
positive.  Pictorial  graphic  organizers  of  both  forms  appear  to  facilitate  the  learning  and 
retention  of  information,  at  least  for  younger  learners.  Learning,  though,  seems  to  have  been 
more  consistently  affected  than  longer-term  retention  of  information.  Taken  overall,  there  is 
positive  evidence  for  the  effectiveness  of  all  three  of  these  techniques.  The  question  posed  by 
this  review,  however,  is  whether  or  not  these  results  generalize  to  instruction  delivered  by 
CBI.  Can  one  expect  any  of  these  techniques  to  help  alleviate  such  potential  problems  as 
disorientation  and  cognitive  overload?  Research  specific  to  the  use  of  instructional  organizers 
in  CBI  is  discussed  next. 


Page  10 


SOS 


Instructional  Organizers  with  CB1 


While  some  of  the  advance  organizer  studies  reviewed  did  use  visual  and  oral 
versions,  the  vast  majority  presented  a  text  form.  Mayer's  characteristics,  however,  indicated 
the  acceptability  of  visual  organizers.  Therefore,  it  is  not  too  large  a  leap  of  faith  to  presume 
that  they  might  be  effectively  incorporated  into  CBI,  a  class  of  technology  which  can 
effectively  combine  both  text  and  visuals.  Hannafin  and  his  associates  have  conducted  most  of 
the  research  on  this  topic,  publishing  a  series  of  papers  on  orienting  activities  in  CBI  generally 
and  CBIV  in  particular  (e.g.  Hannafin,  1987,  Hannafin  &  Hughes,  1986,  Hannafin,  Phillips, 
Rieber,  &  Garhart,  1987).  This  research  is  based  on  the  ROPES+  meta-model  for  designing 
instructional  prescriptions  discussed  above.  In  a  study  on  the  effect  of  orienting  activities  in 
CBI  with  university  students,  Hannafin  and  his  associates  (Hannafin,  Phillips,  Rieber  & 
Garhart,  1987)  found  that  both  behavioral  and  cognitive  orienting  activities  improved  factual 
learning.  Their  description  of  the  cognitive  orienting  activity  was:  "designed  to  provide  an 
integrative  method  for  establishing  meaningful  relationships,  while  also  serving  as  a  subsumer 
of  lesson  detail"  (Hannafin,  Phillips,  Rieber  &  Garhart,  1987,  p.80).  This  sounds  like  an 
advance  organizer.  However,  the  one  example  they  give,  "In  the  next  section,  you  will  be 
presented  information  about:  The  importance  of  studying  cultures."  (Hannafin,  Phillips,  Rieber 
&  Garhart,  1987,  p.77),  seems  insufficient  according  to  Mayer's  characteristics. 

Similarly,  Hannafin,  Phillips  &  Tripp  (1986)  used  a  one  sentence  cognitive  orienting 
activity  in  a  CBIV  lesson  on  artists  and  art  periods  with  80  volunteer  college  students.  They 
noted  a  significant  interaction  between  the  orienting  activity  and  processing  time  but  no  main 
effect.  Another  study  (Hannafin,  1987)  compared  the  effects  of  orienting  activities,  cuing  and 
practice  on  the  learning  of  material  on  space  voyages  by  ninth  grade  students.  A  significant 
interaction  was  found  between  the  cognitive  orienting  activity  and  practice  but  the  orienting 
activity  alone  was  not  demonstrated  to  be  a  significant  instructional  component.  Again,  the 
example  of  the  cognitive  orienting  activity  provided,  'The  Next  Section  Presents  the  Following 
Concepts:  ===>  Unique  lighting  found  throughout  the  solar  system  (and)  ===>  Matter  found 
throughout  the  solar  system",  also  seems  meagre.  Rieber  &  Hannafin  (1988)  also  studied  the 
effects  of  textual  and  animated  cognitive  orienting  activities  on  learning  from  CBI.  Their 
subjects  were  111  students  from  a  rural  elementary  school  presented  with  a  lesson  explaining 
Newton's  laws  of  motion.  Three  types  of  orienting  activities  (of  one  minute  duration  each) 
were  used:  a  text  only,  one  sentence  summary  of  the  particular  basic  concept,  an  animated 
graphic  sequence,  or  a  combined  text  and  graphic  sequence.  There  was  also  a  control  (no 
activity)  group.  The  orienting  activities  were  presented  throughout  the  lesson  before  each 
basic  concept.  They  found  no  statistically  significant  effects  for  any  of  the  orienting  activities. 

The  orienting  activities  in  these  studies  meet  three  of  Mayer's  characteristics  for 
advance  organizer  construction:  (a)  a  short  set  of  verbal  or  visual  information,  (b)  presented 
prior  to  the  material-to-be-learned  and  (c)  containing  no  specific  content  from  that  material. 
However,  one  could  reasonably  argue  as  well  that,  for  none  of  these  studies,  were  the 
cognitive  orienting  activities  of  sufficient  scope  to  meet  the  other  two  conditions;  that  is,  (d) 
generate  logical  relationships  among  elements  in  the  to-be-learned  material,  or  (e)  to 
sufficiently  influence  the  learner's  encoding  process.  The  organizing  activities  used  in  these 
studies  were  not  true  advance  organizers.  Several  studies,  however,  did  use  true  advance 
organizers  in  computer-based  instruction.  One  of  these  studies  was  carried  out  by  Carnes, 
Lindbeck  k  Griffin  (1987)  using  a  computer-based  tutorial  on  kinematics  with  100  suburban 
high  school  physics  students.  The  advance  organizers  were  "written  according  to  the 
guidelines  of  Ausubel  et  al.  (1978)"  (p.785)  and  were  presented  on  screen.  There  was  no 
statistically  significant  difference  between  the  advance  organizer  treatment  group  and  a  non 
organizer  group  (which  read  a  related  passage  designed  not  to  act  as  a  subsumer).  However, 
effect  sizes  of  0.49  for  a  test  of  learning  and  0.14  for  the  retention  test  indicate  a  mild  positive 
effect  by  the  advance  organizer. 
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Krahn  &  Blanchaer  (1986)  tested  the  use  of  a  true  advance  organizer  to  improve 
knowledge  application  by  medical  students  in  a  computer-based  simulation.  Post-test  scores 
showed  a  statistically  significant  difference  between  the  experimental  and  control  group,  both 
for  the  total  scores  and  particular  questions  designed  to  test  far  transfer,  as  predicted  by 
assimilation  encoding  theory.  The  test,  however,  was  given  immediately  after  completion  of 
the  simulation  and  consisted  of  only  six  questions.  No  validity  or  reliability  data  were 
provided.  Insufficient  data  was  reported  to  allow  the  calculation  of  effect  sizes  and,  hence, 
this  study  was  not  included  in  the  previous  table  (Table  2). 

Tripp  &  Roby  (1990, 1991)  reported  the  use  of  an  advance  organizer  in  two  studies 
with  a  Japanese-English  hypertext-based  lexicon.  The  measure  of  learning  in  these 
investigations  was  of  immediate  recall  and  was,  in  fact,  an  assessment  of  what  the 
investigators  viewed  as  rote  learning.  The  authors  reasoned,  however,  that  the  advance 
organizer  would  convey  the  structure  of  the  database  and,  therefore,  contribute  to  meaningful 
learning.  Results  of  the  first  study  indicated  a  probability  coefficient  of  p  <  0.171. 
Interestingly,  despite  the  low  p  coefficient,  the  effect  size  for  the  organizer  was  1.25,  which 
appears  to  indicate  a  lack  of  power  in  the  study.  However,  half  of  the  group  using  the 
advance  organizer  also  received  a  visual  metaphor.  A  statistically  significant  negative  effect 
was  reported  for  the  interaction  between  the  two  techniques.  Tripp  and  Roby  speculated  that 
otherwise  the  advance  organizer  could  have  been  expected  to  have  a  statistically  significant 
effect  upon  learning. 

In  the  second  study,  the  advance  organizer  was  rewritten  to  provide  a  metaphorical 
structure  hypothesized  to  be  congruent  with  the  visual  metaphor  treatment.  The  authors 
reported  a  significant  main  effect  for  the  advance  organizer  treatment  but  not  for  the  visual 
metaphor,  nor  was  there  a  significant  interaction.  Comparison  of  the  advance  organizer  only 
treatment  group  mean  to  the  control  group  mean,  however,  reveals  a  much  smaller  effect  size 
(033)  than  the  first  study.  The  second  study,  though,  used  a  considerably  larger  number  of 
subjects  and,  with  the  increased  power,  may  have  produced  a  more  accurate  result 
Regardless,  these  studies  clearly  contradict  advance  organizer  theory  since  they  demonstrated 
that  the.  technique  facilitated  rote,  rather  than  meaningful,  learning. 

Kenny,  Grabowski,  Middlemiss  k  Van  Neste-Kenny  (1991)  compared  the  effects  of 
participatory  pictorial  graphic  organizers  to  those  of  the  identical  final  form  versions  on  the 
learning  of  third  year  nursing  students  from  a  CBIV  program  on  nursing  elderly  patients  with 
chronic  obstructive  pulmonary  disease.  The  group  receiving  the  participatory  graphic 
organizer  substantially  outperformed  the  final  form  group  on  a  test  of  learning,  scoring  an 
average  of  1.77  points  higher  on  an  18  question  multiple  choice  test  (ES.  *  059).  The 
difference,  however,  was  not  statistically  significant.  As  well,  there  was  only  a  very  slight 
difference  between  the  two  treatments  on  the  retention  test  (given  one  week  later)  and  in 
favour  of  the  final  form  version  (E.S.  =  -0.07).  Unfortunately,  considerable  unanticipated 
extraneous  note-taking  by  subjects  in  both  groups  confounded  the  differences  in  generativity 
between  the  two  treatments.  One  interesting,  though  not  statistically  significant,  result  was 
that  the  final  form  graphic  organizer  group  scored  almost  two  points  higher  (on  an  18 
question,  multiple  choice  test)  on  the  retention  test  than  on  the  test  of  learning.  This  may 
provide  evidence  that  a  final  form  graphic  organizer  behaves  in  a  subsumptive  manner  akin  to 
that  posited  for  advance  organizers. 

In  a  subsequent  study,  Kenny  (1992)  then  compared  the  use  of  an  advance  organizer 
to  that  of  participatory  and  final  form  graphic  organizers  with  a  CBIV  on  cardiac  nursing.  In 
this  study,  it  was  the  final  form  graphic  organizer  which  was  clearly  the  most  effective 
treatment,  garnering  the  highest  mean  scores  on  both  tests  of  learning  and  retention. 
Curiously,  the  participatory  graphic  organizer  group  had  the  lowest  mean  scores  while  the 
advance  organizer  group  fell  in  the  middle.  The  difference  between  the  final  form  and 
advance  organizer  group  means  was  statistically  significant  at  the  p  £  0.05  level  for  learning 
(E.S.  =  1.17)  but  was  not  significant  for  retention  (ES.  =*  0.45).  The  difference  between  the 
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advance  and  participatory  organizer  group  means  was  statistically  significant  at  the  p  s  0.10 
level  for  retention  (E.S.  =  •  0.95)  but  not  significant  for  the  test  of  learning  (E.S.  =  -  0.45).  The 
effect  sizes  reported  here  (and  in  Table  4)  assume  the  advance  organizer  as  a  control. 
Extraneous  note-taking  was  controlled.  However,  the  cardiac  program  used  a  guided 
discovery  approach  design  (unlike  the  pulmonary  program)  which  demanded  considerable 
interaction  on  the  part  of  the  learner  and  may  have  interacted  with  the  organizer  treatments 
(See  the  section  'The  Generative  Learning  Hypothesis"). 

Based  on  the  results  of  these  studies,  there  appears  to  be  mild  evidence  to  suggest  that 
advance  organizers  and  pictorial  graphic  organizers  could  be  effective  if  incorporated  in 
instruction  based  on  CBI.  Furthermore,  this  literature  is  congruous  with  the  research  on  the 
use  of  these  techniques  in  instruction  in  general,  that  is,  the  evidence  of  their  effectiveness  is 
mostly  positive,  if  somewhat  conflicting.  Why  are  these  generally  positive  results  not  reflected 
more  often  in  the  research?  In  the  case  of  the  studies  focusing  on  CBI,  many  of  the  organizers 
did  not  appear  to  be  properly  constructed,  that  is,  they  were  insufficient  to  produce  a 
subsumptive  effect.  Again  this  may  reflect  the  theme  sound,  ^  by  McEneany  (1990)  that  a 
sound  operational  definition  for  the  construction  of  advance  organizers  is  lacking.  The  format 
and  construction  of  the  pictorial  graphic  organizer  is  similarly  unclear  and  often  (in  the 
experience  of  the  author)  difficult.  Perhaps  too,  however,  there  is  a  problem  with  the 
underlying  theory.  If  subsumption  theory  and  assimilation  encoding  theory  are  not  effective 
in  predicting  when  instructional  organizers  will  be  effective,  will  another  theory  be  more 
accurate?   In  a  discusaion  of  the  psychological  underpinnings  of  hypermedia,  Borsook  & 
Higginbotham-Wheat  (1992,  p.62)  note  that  as  "we  explore  some  of  the  ideas  of  what  we  know 
about  how  we  learn  and  apply  knowledge,  it  becomes  obvious  that  activity  as  well  as 
interactivity  [emphasis  added]  are  integral  components  of  both  theory  and  its  application  in  the 
technology  of  hypermedia."  Cognitive  principles  suggest  that  learning  is  an  active, 
constructive  process  in  which  learners  generate  meaning  for  information  by  accessing  and 
applying  existing  knowledge  (Borsook  &  Higginbotham-Wheat  (1992,  p.64).  They  point  out 
that  Wittrock's  (1974)  generative  learning  theory  incorporates  such  principles.  This  theory  is 
considered  next. 

The  Generative  Learning  Hypothesis 

In  Wittrock's  (1974,  p.88)  view  ,  "it  is  the  learner's  interpretation  of  and  processing  of 
the  stimuli,  not  their  [the  stimuli's]  nominal  characteristics,  which  is  primary").  Learners  must 
construct  their  own  meaning  from  teaching  (Wittrock,  1985).  This  meaning  is  generated  by 
activating  and  altering  existing  knowledge  structures  to  interpret  new  information  and  encode 
it  effectively  for  future  retrieval  and  use.  Further,  generative  learning  involves  not  only 
generating  meaning,  but  overt  activities  as  well,  such  as  generating  associations  among  words, 
generating  pictures,  etc.  (Doctorow,  Wittrock  &  Marks,  1978).  These  learning  activities  require 
the  learner  to  relate  new  information  to  an  existing  knowledge  structure  and  depend  on 
complex  cognitive  transformations  and  elaborations  that  are  individual,  personal  and 
contextual  in  nature.  Information  is  transformed  and  elaborated  into  a  more  individual  form 
making  it  more  memorable  as  well  as  more  comprehensible  (Borsook  &  Hieeinbotham-Wheat 
1992). 

Advance  organizers  were  developed  from  Ausubel's  theories  about  meaningful  verbal 
learning  while  Mayer  analyzed  the  research  in  terms  of  assimilation  encoding  theory.  Pictorial 
graphic  organizers,  on  the  other  hand,  while  a  derivative  of  advance  organizers,  are  neither 
based  on  subsumption  nor  assimilation  encoding  theory.  In  fact,  Hawk,  McLeod  &  Jonassen 
(1985)  recommended  the  participatory  version  because  it  was  their  belief  that  "the  more 
generative  the  nature  of  student  participation,  the  more  likely  it  is  that  transfer  and  higher 
level  learning  will  be  affected"  (p.  179).  They  are  referring  here  to  Wittrock's  (1974)  generative 


Page  13 


ERLC 


508        c  • 


learning  hypothesis.  Participatory  graphic  organizers  elicit  generative  activity  because  they 
require  the  learner  to  actively  search  a  body  of  material  to  select  information  to  complete  the 
organizer.  True  advance  organizers,  on  the  other  hand,  cannot  necessarily  be  considered  to  be 
generative  activities.  They  invoke  a  covert  response  on  the  part  of  learners  which  may  or  may 
not  be  generative,  depending  on  whether  or  not  they  generate  meaning  between  the  organizer 
and  their  prior  knowledge  and  between  the  organizer  and  the  material-to-be-learned.  The 
advance  organizer  does  not  engage  the  learner  in  overt,  active  learning.  Of  the  instructional 
organizers  considered  thus  far,  therefore,  only  the  participatory  form  of  the  pictorial  graphic 
organizer  advanced  by  Hawk,  McLeod  &  Jonassen  (1985)  and  the  student-constructed  form  of 
structured  overview  (Moore  &  Readance,  1984)  could  be  considered,  by  design,  to  elicit  a 
generative  response  from  the  learner.  Perhaps  this  explains  why  the  research  on  instructional 
organizers,  and  particularly  advance  organizers,  is  so  variable.  Given  the  difficulty  in 
determining  exactly  how  to  construct  them,  it  may  be  that  some  investigators  unknowingly 
designed  their  organizers  or  some  other  aspect  of  their  studies  such  that  the  learners  engaged 
in  generative  learning  activities. 

In  fact,  Moore  &  Readance  (1984)  note  superior  results  for  structured  overview  graphic 
organizers  constructed  after  reading  the  learning  passage.  In  support  of  Wittrock's  hypothesis 
that  familiar  words  facilitate  the  learners'  generation  of  meaning  for  the  passage  (Marks, 
Doctorow  &  Wittrock,  1974,  Wittrock,  Marks  &  Doctorow,  1975),  Corkhill,  Bruning  and  Glover 
(1988)  demonstrated  the  advantage  of  concrete  advance  organizers  over  more  abstract  ones, 
suggesting  that  the  former  furnish  ideational  anchorage  in  terms  already  familiar  to  the 
learners.  Even  the  Alvermann  (1988)  study  of  final  form  graphic  organizers  directed  the 
learners  to  engage  in  what  can  be  argued  to  be  a  generative  activity  by  asking  them  to  use  the 
organizer  as  a  map  in  an  active  search  back  in  the  text  for  question  answers.  Thus,  some  of 
the  most  impressive  results  were  garnered  when  students  were  actively  engaged  in  the 
learning  process. 

Generative  instructional  organizers  with  CBI 

Generative  learning  theory  as  applied  to  CBI,  at  least  as  pertains  to  instructional 
organizers,  has  yet  to  be  widely  tested.  The  few  studies  completed  have  not  provided  strong 
evidence  to  suggest  that  generative  learning  activities  can  be  successfully  applied  to  computer- 
based  media.  Two  studies  described  earlier  (Kenny,  Grabowski,  Middlemiss  &  Van  Neste- 
Kenny,  1991,  Kenny,  1992)  obtained  mixed  results.  Kenny,  Grabowski,  Middlemiss  &  Van 
Neste-Kenny  (1991)  compared  the  use  of  final  form  and  participatory  graphic  organizers  with 
CBIV  and  found  a  mild  effect  size  in  favour  of  the  participatory  pictorial  graphic  organizer. 
Kenny  (1992),  however,  compared  nearly  identical  participatory  and  final  form  graphic 
organizers  to  an  advance  organizer,  again  with  CBIV,  and  found  the  final  form  graphic 
organizer  to  be  the  most  effective  on  both  measures  of  learning  and  retention.  The  group 
using  the  participatory  organizer,  hypothesized  to  be  the  technique  most  likely  to  elicit 
generative  learning,  achieved  the  lowest  mean  scores.  However,  analysis  of  interview  data 
indicated  that  the  guided  discovery  design  of  the  CBIV  program  may  have  interfered  with  the 
normally  generative  nature  of  the  participatory  graphic  organizer.  In  effect,  the  learners  were 
already  engaged  in  a  generative  learning  activity,  one  which  was  quite  demanding.  In  this 
situation,  the  normally  generative  organizer,  rather  than  helping  to  make  learning  more 
meaningful,  likely  contributed  to,  rather  than  alleviated,  cognitive  overload. 

In  another  study  with  tutorial  courseware  delivered  by  CBIV,  Harris  (1992)  compared 
the  use  of  learner-generated  summaries  to  the  completion  of  multiple  choice  questions. 
Contrary  to  predictions,  the  control  group,  which  received  the  treatment  considered  to  be  least 
generative  (multiple  choice  questions  completed  at  the  end  of  each  module),  achieved  the 
highest  mean  score  on  a  test  of  learning  (given  right  after  completion  of  the  modules).  Effect 
sizes  were  -  0.44  for  the  learner-generated  summaries  without  feedback  and  -0.12  for  those 
with  feedback.  None  of  the  differences  were  statistically  significant.  Finally,  Jonassen  &  Wang 
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(1992),  conducted  three  studies  on  acquiring  structural  knowledge  from  hypertext.  Structural 
knowledge  is  that  of  how  concepts  in  a  particular  domain  are  interrelated  (Jonassen  k  Co'e 
1992).  In  one  study  (the  second),  Jonassen  k  Wang  (1992)  tested  the  use  of  a  generative   '  ' 
activity  with  the  HyperCard  version  of  their  text.  Hypertext  /  HvpermpHi*  The  control 
treatment  consisted  of  referential  links  embedded  in  the  cards,  while  the  generative  treatment 
asked  the  learners  to  classify  the  nature  of  the  relationship  between  the  node  they  were 
leaving  and  the  one  they  were  traversing  to.  As  with  the  Harris  study,  the  control  treatment 
group  was  generally  more  successful,  scoring  higher  on  the  average,  on  a  test  of  recall  (E  S  =  - 
0.64)  and  on  2  of  3  tests  of  structural  knowledge  (E.S.  for  relationship  proximity  =  036  E  S  for 
semantic  relationships  =  0.43  and  E.S.  for  analogies  -  -  0.15).  Again,  none  of  the  differences 
were  statistically  significant. 

Clearly,  these  studies  are  far  from  conclusive.  Three  of  these  four  studies  use  small 
sample  sizes  and  may  have  been  underpowered.  Only  the  first  provided  any  evidence  for  the 
effectiveness  of  generative  activities  in  CBI.  While,  theoretically,  the  application  of  generative 
learning  theory  to  the  use  of  instructional  organizers  with  CBI  seems  to  hold  promise,  there 
has  been  little  evidence  to  demonstrate  that  it  more  effectively  predicts  the  effectiveness  of 
instructional  orienting  activities  than  the  theories  considered  previously.  Is  there  any  theory 
that  will  provide  guidance?  In  fact,  Borsook  k  Higginbotham-Wheat  (1992)  suggest  that  a 
number  of  theories,  generative  learning  among  them,  may  provide  insight  about  how  and  why 
hypermedia  may  work.  Perhaps  most  prominent  among  these  cognitive  theories  is  schema 
theory  (Rumelnart  k  Orteny,  1977,  Anderson,  1977). 

Schema  Theory  and  the  Hypermap 

Jonassen  (1989)  claimed  that  it  is  schema  theory  that  describes  the  organization  of 
human  memory,  not  Ausubel's  hierarchical,  or  subsumptive,  model.  A  schema  for  an  object 
event  or  idea  is  comprised  of  a  set  of  attributes,  that  is,  associations  that  one  forms  around  an 
idea,  schema  are  in  turn  arranged  into  semantic  networks,  sets  of  nodes  with  ordered 
relationships  connecting  them  (Jonassen,  1988).  Kiewra  (1988)  indicates  that  an  outgrowth  of 
schema  theory  has  been  applied  research  on  the  effectiveness  of  spatial  learning  strategies 
Such  strategies  involve  the  reorganization  of  information  into  some  form  of  spatial 
representation  that  clarifies  the  relationships  among  inherent  ideas  or  concepts  Such 
representations  allow  information  to  be  more  readily  processed  since  they  reflect  cognitive 
structure  and  provide  multiple  retrieval  cues  for  accessing  it.  Since  the  graphic  organizer  is 
one  form  of  spatial  representation,  schema  theory  may  help  explain  the  effectiveness  of  the 
technique.  In  the  study  by  Kenny  (1992),  for  instance,  the  filled-in  (Final  Form)  version  of  the 
pictorial  graphic  organizer  was  most  effective  and  may  have  acted  as  a  form  of  cognitive  map 

To  reflect  an  individual's  knowledge  structure,  then,  an  instructional  organizer,  rather 
tnanbeing  at  a  higher,  more  abstract  level  as  is  an  advance  organizer,  might  be  designed  to 
reflect  either  novice  or  expert  schemata,  particularly  if  the  hypermedia  is  unstructured.  To  do 
this,  Jonassen  (1989),  advocated  the  use  of  hypermaps,  or  graphical  browsers,  an  instructional 
orienting  technique  similar  to  the  graphic  organizer.  A  hypermap  provides  a  graphical  view 
of  the  program  structure.  The  user  may  select  a  node  on  the  hypermap  and  be  taken 
immediately  to  that  part  of  the  program.  Hypermaps  represent  a  graphical  interface  between 
oLU^an  3  bpe*™1  that  is  designed  to  reduce  navigation  problems  (Jonassen  k  Wang, 
1992).  They  can  be  expected  to  be  effective  because  they  should  enhance  the  learner's 
structural  knowledge  (the  knowledge  of  interrelationships  between  ideas)  of  the  information  in 
the  program  (Jonassen,  1989). 

Research  on  hypermaps,  however,  has  been  even  more  scant  than  that  on  generative 
techniques  in  CBI.  In  one  of  the  studies  described  above  (the  first),  Jonassen  k  Wang  (1992) 
C0H!SL™.*e  <"ffectiveness  of «  hypermap  to  a  control  treatment  consisting  of  referential  links 
embedded  in  the  cards.  As  with  the  other  study,  the  control  group  outperformed  the 
hypermap  group  on  a  recall  test.  The  difference  was  statistically  significant.  Tests  of  structural 
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knowledge  acquisition,  however,  showed  no  significant  differences  between  the  techniques.  In 
the  third  study,  Jonassen  k  Wang  (1992)  provided  learners  with  either  a  control  treatment  as 
described  above  or  a  graphical  browser  (or  hypermap)  to  use  for  navigation.  They  informed 
half  the  learners  in  each  group  that  they  would  be  responsible  for  developing  a  semantic 
network  (essentially  their  own  hypermap)  after  completing  the  program.  While  this  activity 
was  ceased  after  only  a  few  minutes  in  order  to  control  for  time-on-task,  those  given  the 
semantic  network  task  performed  significantly  better  on  the  semantic  relationships  scale  and 
the  graphical  browser/semantic  network  group  was  significantly  better  on  the  analogies 
subscale.  In  effect,  the  instructions  to  the  semantic  networks  groups  may  have  been  sufficient 
to  lead  them  to  actively  engage  the  material,  that  is,  to  elicit  generative  learning.  The 
analogies  subscale  result,  then,  is  one  that  could  be  explained  by  either  Wittrock's  (1974) 
ftcory,  by  schema  theory,  or  both.  These  two  studies  appear  to  be  the  only  ones  to  date 
testing  this  form  of  orienting  activity,  although  a  variation  of  the  TCU  knowledge  mapping 
system  (e.g.  Dansereau  et  al.,  1979)  has  been  tested. 

Like  the  graphical  browser,  the  TCU  knowledge  map  represents  multi-dimensional 
knowledge  in  associative  networks  akin  to  semantic  nets  or  schemata.  Nodes  denote  the  type 
and  importance  of  the  content  and  the  spatial  properties  of  the  map  clarify  the  organization  of 
the  domain  (Reynolds  and  Dansereau,  1990).  Perhaps  the  greatest  distinguishing  feature  of  the 
TCU  knowledge  map  is  the  use  of  8  specified  link  types  to  label  linkages  between  nodes  while 
Jonassen's  version  uses  flexible  labelling  for  links.  Two  recent  studies  (Reynolds  and 
Dansereau,  1990,  Reynolds  et  al.,  1991)  have  presented  this  technique  to  learners  in 
computerized  form  as  a  hypermap.  Two  variations  of  a  statistics  package  were  developed:  a 
"standard"  hypertext  version  and  a  version  in  which  all  the  concepts  were  represented  in 
hypermap  versions  of  the  knowledge  map,  that  is,  no  standard  text  screens  were  provided. 
Since  these  studies  used  this  format  for  the  main  body  of  the  learning  material  rather  than  as 
an  orienting  activity,  these  studies  are  not  reviewed  here. 

Conclusion 

As  has  been  seen,  the  research  pertaining  to  the  use  of  instructional  organizers  in  CBI 
has  not  been  extensive.  Much  of  it  has  been  carried  out  under  the  umbrella  of  the  ROPES+ 
meta-model  proposed  by  Hannafin  and  his  associates.  While  advance  and  graphic  organizers 
are  included  in  the  orienting  activities  category,  it  is  doubtful  that  the  cognitive  orienting 
activities  used  by  these  theorists  (e.g.  Hannafin,  1987,  Hannafin  &  Hughes,  1986,  Hannafin, 
Phillips,  Rieber,  &  Garhart,  1987)  have  represented  forms  of  these  techniques.  Where  such 
organizers  have  been  constructed  according  to  original  guidelines,  they  have  been  somewhat 
more  successful.  Yet  the  evidence  has  not  been  consistent,  a  result  which  is  perhaps  not 
surprising  given  the  modest  results  reported  in  past  research.  However,  neither  has  the 
limited  research  testing  instructional  organizers  on  the  basis  of  two  other  theories,  generative 
learning  theory  and  schema  theory,  been  strongly  indicative  of  their  predictive  value.  Based 
on  this  review,  it  may  be  that  no  one  theory  is  sufficient  in  and  of  itself  to  predict  the 
effectiveness  of  instructional  orienting  techniques  and  to  guide  their  design.  Rather,  it  may  be 
necessary  to  consider  various  theories  acting  in  concert.  As  discussed  previously,  Borsook  & 
Higginbotham-Wheat  (1992)  have  suggested  that  a  number  of  theories,  generative  learning  and 
schema  theory  among  them,  may  provide  insight  about  how  and  why  hypermedia  may  work. 
If  the  results  of  the  third  Jonassen  «nd  Wang  (1992)  study,  which  combined  a  hypermap 
treatment  with  a  generative  activity,  are  indicative,  the  design  of  instructional  organizers  may 
have  to  be  based  on  features  from  various  theories,  each  of  which  describes  a  different  aspect 
of  human  cognition.  While  further  research  on  both  the  application  of  generative  learning  and 
schema  theory  to  CBI  is  needed,  perhaps  this  will  be  most  fruitful  if  It  combines  the  two. 
Simple  tests  of  theory  may  not  be  sufficient. 
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ABSTRACT 


Educators,  administrators,  and  students  are  re-evaluating  the  value  of  animal 
dissection  in  the  classroom  and  are  taking  a  careful  look  at  instructional  alternatives. 
This  research  is  an  attempt  to  examine  the  performance,  achievement,  and  atiiludlnal 
effects  of  a  dissection  alternative,  an  interactive  videodisc-based  (IVD)  simulation, 
in  two  ways:  as  a  substitute  for  dissection  and  as  a  preparatory  tool  used  prior  to 
dissection.  Sixty-one  high  school  students  enrolled  in  three  general  ability  high 
school  biology  classes  participated  in  this  research  over  a  four  day  period  On  the 
substitution  issue,  findings  suggest  that  the  IVD  simulation  was  at  least  as  effective 
as  actual  dissection  in  promoting  student  learning  of  frog  anatomy  and  dissection 
procedures.  On  the  preparation  issue,  it  was  found  that  students  using  the  IVD 
simulation  as  a  preparation  performed  a  subsequent  dissection  more  effectively  than 
students  receiving  no  preparation  and  more  effectively  than  students  viewing  a 
videotape  as  preparation,  Students  using  the  IVD  simulation  as  preparation  also 
learned  more  about  frog  anatomy  and  dissection  procedures  than  those  who 
dissected  without  preparation.  Students  in  all  groups  evidenced  little  change  in 
attitudes  towards  dissection.  All  students  reported  a  significant  gain  in  dissection 
self -efficacy,  but  no  between  group  differences  were  found.  Findings  are 
discussed  relative  to  their  implications  for  educational  practice  and  future  research. 
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The  Effect*  of  an  Interactive  Dissection  Simulation 
on  the  Performance  and  Achievement  of  High  School 
Biology  Students 

Frog  dissection  in  our  nation's  high  schools 
is  widespread.  It  has  been  estimated  that  75  to  80% 
of  the  country's  four  million  biology  students  dissect 
frogs  (Orlans,  1988a).  As  part  of  a  growing 
controversy  over  the  use  of  animals  for  dissection, 
some  of  these  students  are  refusing  to  conduct 
dissections  on  moral  grounds.  Animal  rights  groups 
have  developed  student  and  educator  outreach 
programs  encouraging  alternatives  to  dissection. 
Legislation  has  been  passed  in  some  states  protecting 
the  rights  of  students  who  do  not  wish  to  participate 
in  dissection.  Many  educators  contend,  however,  that 
there  is  merit  in  conducting  dissections  to  assist 
students  in  learning  about  the  anatomy  and  biological 
functioning  of  animals. 

The  research  repotted  here  is  an  attempt  to 
examine  the  effects  of  a  dissection  alternative,  a 
videodisc-based  simulation,  in  two  ways.  Fust, 
because  it  is  of  interest  to  determine  if  such  an 
alternative  could  be  as  effective  a  learning  tool  as 
actual  dissection,  the  effects  of  the  simulation  were 
compared  to  those  of  an  actual  dissection.  Relevant 
outcomes  were  student  achievement  ona  test  of  frog 
anatomy  and  dissection  procedures  and  dissection- 
related  attitudes  and  self  efficacy.  Second,  because 
many  educators  believe  in  the  value  of  dissection,  the 
simulation  was  examined  as  a  preparatory  tool,  used 
prior  to  dissection.  Here  the  achievement,  attitude, 
and  self-efficacy  scores  were  also  employed,  along 
with  a  measure  of  students1  actual  dissection 
performance.  Comparisons  were  made  between  the 
following  groups:  (a)  students  receiving  the 
interactive  videodisc-based  simulation  as  preparation, 
(b)  students  viewing  a  linear  videotape  (containing  the 
same  video  images  as  the  simulation  but  without  the 
interactive  practice)  as  preparation,  and  (c)  students 
receiving  no  preparation. 

In  this  paper,  we  briefly  outline  the 
controversy  surrounding  dissection.  We  go  on  to 
describe  the  design  of  the  videodisc  simulation, 
followed  by  a  presentation  of  research  methods  and 


results.  Finally,  the  results  will  be  discussed  with 
regard  to  their  implications  for  educational  practice 
and  future  research. 

Origins  of  Dissection  in  ita  Bioinpy  riayjmnm  nT1f1 

the  Currenr  Cnnfmv^y 

Animals  have  been  dissected  in  biology 
classes  since  the  early  1900's.  According  to  Orlans 
(1988a)  it  is  believed  that  dead  frogs  first  became 
available  from  commercial  suppliers  between  1910 
and  1920.  Even  though  frog  dissection  was  not 
included  in  the  first  formal  biology  textbook 
published  in  1921,  it  had  become  an  established 
laboratory  activity  in  the  nation's  high  schools  by  the 
1920*.  By  the  1960's,  frog  dissection  had  become 
very  popular  as  a  result  of  the  new  biology  curricula 
developed  by  the  Biological  Sciences  Curriculum 
Study  (BSCS)  (Hairston,  1990).  As  mentioned 
previously,  by  1988  it  was  estimated  that  75  to  80% 
of  this  country's  four  million  biology  students  were 
dissecting  frogs  (Orlans,  1988a). 

In  recent  years  animal  dissection  has  come 
under  increased  scrutiny.  As  a  result  of  several 
judicial  and  legislative  decisions,  students,  teachers, 
and  administrators  are  being  compelled  to  reevaluate 
the  morality  and  instructional  effectiveness  of  this  use 
of  animals.  The  California  and  Florida  legislatures 
passed  bills  protecting  the  rights  of  students  who  do 
not  wish  to  participate  in  dissection  (Orlans,  1988a). 
Massachusetts  considered  similar  legislation  in  1990 
(Smith,  1990). 

Some  science  educators  and  scholars,  such  as 
Orlans  (1988a,  1988b),  advocate  replacing  the 
traditional  dissection  lab  with  the  study  of  live 
organisms.  Others,  such  as  Bennan  (1984),  Hoskins 
(1979),  and  Igelsrud  (1986, 1987),  express  support  for 
dissection.  Their  support,  however,  is  conditional 
upon  the  use  of  great  care  and  planning  in  the  design 
of  lessons  using  animal  specimens. 

The  National  Association  of  Biology 
Teachers  (NABT)  policy  statement,  "Animals  in 
Biology  Classrooms,"  recognizes  the  long  history  of 
animal  dissection  in  biology  education  and  the 
educational  value  of  "well  constructed  dissection 
activities  conducted  by  thoughtful  instructors" 


:RLC 


526 


Interactive  Dissection  Simulation 


(Haireton,  1990,  p.  x.)  Nevertheless,  the  NABT 
statement  suggests  that  biology  teachers  reexamine 
their  reasons  for  including  animal  dissection  in  their 
courses  and  consider  using  alternatives. 

Currently  available  alternatives  include 
books,  charts,  computer  programs,  models, 
filmstrips,  slides,  transparencies,  videotapes,  and 
videodiscs.  Many  of  these,  however,  can  be  faulted 
for  their  lack  of  realism  and  opportunities  for  student 
involvement  Furthermore,  little  research  has  been 
conducted  to  evaluate  their  effectiveness.  Film  or 
video-based  alternatives  have  the  greatest  potential  to 
accurately  depict  the  animal  being  studied  since  they 
show  actual  specimens.  Results  of  research 
comparing  the  use  of  two  film-based  alternatives  to 
actual  dissection  in  medical  and  veterinary  anatomy 
instruction  suggest  that  such  alternatives  can  be  as 
effective  as  dissection  in  promoting  cognitive 
outcomes  (Prentice  ct  al.,  1977;  Wclscr.  1969). 

The  Interactive  Frop  Dissection 

Since  the  film-based  dissection  alternatives 
mentioned  above  were  evaluattd*  interactive  videodisc 
technology  (IVD)  has  emerged  as  an  instructional  tool 
of  great  potential  for  instructional  simulations.  Iuppa 
(1984)  defines  interactive  video  as  Nany  video  system 
in  which  the  sequence  and  selection  of  messages  is 
determined  by  the  user's  response  to  the  material**  (p. 
5).  This  feature,  which  allows  learners  to  experience 
consequences  to  their  decisions,  is  probably  the  most 
significant  feature  of  IVD.  IVD-based  simulations 
take  advantage  of  the  medium's  high  quality  audio  and 
video,  instant  random  access  capability  (any  location 
on  a  videodisc  can  be  accessed  in  two  seconds  or  less), 
playback  flexibility  (e.g.,  variable  speed  and  direction, 
different  audio  tracks  depending  on  user  performance), 
and  storage  capacity  (up  to  54,000  still  images  or  30 
minutes  full-motion  video).  These  features  are  then 
coupled  with  a  microcomputer's  text,  graphics,  and 
videodisc  control  capabilities. 

An  IVD-based  simulation  of  frog  dissection, 
called  The  Interactive  Frog  Dissection  was  developed 
by  Strauss  and  Kinzie  (1991)  as  a  research  tool  to 
study  the  potential  of  dissection  alternatives.  It 
includes  demonstrations  of  frog  dissection  and 
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depictions  of  frog  anatomy  interspersed  with  practice 
activities  on  dissection  procedures  and  anatomical 
identification.  Additional  directions,  explanations, 
and  feedback  are  supplied  in  response  to  student  input, 
ensuring  a  high  degree  of  interactivity. 

Results  from  pilot  research  suggest  that 
students  using  The  Interactive  Frog  Dissection  learn 
as  much  about  dissection  procedures  and  frog  anatomy 
as  students  conducting  an  actual  dissection  (Strauss  & 
Kinzie,  in  press).  Hie  following  study  continues  this 
investigation,  comparing  the  achievement  and 
attitudinal  effects  of  the  simulation  to  those  of 
laboratory  dissection,  and  extending  the  exploration  to 
include  the  use  of  the  simulation  as  a  preparatory  tool 
prior  to  dissection. 

METHOD 

Subjects: 

Participating  in  this  research  were  61 
students  (33  male  and  28  female)  at  a  high  school 
located  near  a  small  mid-Atlantic  city.  The 
participants  were  an  average  age  of  13.30  (SD  -  0.92) 
and  were  enrolled  in  three  general  ability  biology 
classes  taught  by  a  single  instructor.  Seventy-seven 
percent  (n  *  47)  reported  participating  in  a  previous 
dissection,  all  but  one  occurring  in  the  seventh  grade. 
The  study  was  conducted  over  four  consecutive  days 
following  a  unit  on  human  anatomy. 

Procedures: 

Students  in  each  of  the  three  classes  were 
randomly  assigned  to  one  of  four  groups:  (1)  the  IVD 
Prep  group  used  the  interactive  videodisc-based 
simulation  as  a  preparation  for  laboratory  dissection 
(n  ■  16);  (2)  the  Video  Prep  group  viewed  a  linear 
videotape  containing  the  same  video  materials  used  in 
the  IVD  simulation,  but  without  interactivity  (n  • 
IS);  (3)  the  Diss  Only  group  conducted  the  dissection 
without  preparation  (n  •  IS);  and  (4)  the  IVD  Only 
group  used  the  IVD  simulation  but  did  not  dissect  (n 
»  IS).  Within  each  group,  students  were  randomly 
assigned  to  teams  of  two  or  three  each.  Preliminary 
analyses  indicated  that  there  were  no  significant 
between-group  differences  on  the  biology  course  grade 
earned  prior  to  that  point  in  the  semester. 
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On  the  first  day  of  the  study,  the  teacher  told 
the  students  that  they  would  be  helping  to  evaluate 
some  instructional  activities  related  to  dissection. 
The  teacher  explained  that  because  there  were  not 
enough  computers  for  everyone  to  use  them  at  once, 
they  would  be  splitting  up  into  groups,  with  different 
groups  sometimes  engaging  in  different  activities. 
Students  were  also  informed  that  everyone  would  have 
the  opportunity  to  use  the  computers.  (Even  students 
whose  instructional  treatment  did  not  include  use  of 
the  computer-based  simulation  were  given  the 
opportunity  to  use  the  simulation  after  completion  of 
data  collection.)  Students  then  completed  the 
achievement  pretest  and  attitude  and  self-efficacy  pre- 
measures.  There  were  no  significant  differences 
between  treatment  groups  on  pretest  achievement, 
attitude,  or  self-efficacy, 

On  day  two,  students  in  the  IVD  Prep  group 
used  the  simulation,  students  in  the  Video  Prep  group 
viewed  the  videotape,  and  students  in  the  Diss  Only 
or  IVD  Only  groups  completed  library  research  for  an 
unrelated  biology  assignment.  For  their  use  of  the 
simulation,  students  in  the  IVD  Prep  group  broke 
into  their  randomly  assigned  teams  (two  teams  per 
class  period).  Each  team  worked  through  the 
simulation  at  one  of  two  IVD  stations  located  in  their 
classroom.  IVD  Prep  students  spent  an  average  of 
39.4  minutes  (SD  *  3.6)  on  the  dissection 
simulation.  Students  in  the  Video  Prep  group  went 
to  a  nearby  classroom  to  view  the  videotape,  which 
lasted  15  minutes.  On  completion  of  the  videotape, 
the  Video  Prep  students  went  to  the  library  to  work 
on  an  unrelated  biology  assignment. 

During  day  three,  student  teams  in  the  IVD 
Prep,  Video  Prep,  or  Diss  Only  groups  completed  a 
frog  dissection  laboratory  at  laboratory  benches  in 
their  classroom.  Students  in  the  IVD  Only  group 
broke  into  their  two  teams  and  used  the  interactive 
videodisc-based  simulation.  The  IVD  stations  were 
placed  in  corners  of  the  classroom  some  distance  away 
from  the  laboratory  benches  and  the  other  students. 
There  was  very  little  to  no  interaction  between 
students  conducting  the  dissection  and  students  using 
the  simulation.  All  students  were  engaged  in  their 
respective  activities  for  the  duration  of  the  class 


period.  On  the  fourth  and  final  day,  students  in  all 
groups  completed  the  achievement  post-test  and  the 
attitude  and  self-efficacy  post-measure.  Students  were 
told  that  their  grade  on  the  posttest  would  count 
towards  their  course  grade,  so  that  they  should  try 
hard  to  do  well.  On  day  four,  one  student  was  absent 
and  was  dropped  from  all  analyses  involving  any  of 
the  post-measures.  Another  student  was  called  out  of 
class  during  the  posttest;  this  student  was  dropped 
from  analyses  involving  the  posttest 

Materia]  ft; 

Students  in  the  IVD  Prep  and  IVD  Only 
groups  used  the  interactive  videodisc-based  simulation 
previously  described  The  videotape  used  in  Video 
Prep  treatment  was  taken  from  the  simulation 
videodisc.  It  presents  and  reviews  dissection 
proccdures'iften  presents  and  reviews  frog  anatomy, 
but  differs  from  the  IVD  materials  in  that  it  does  not 
provide  opportunities  for  practice. 

Student  teams  conducting  the  dissection  on 
day  three  received  a  lab  handout  detailing  dissection 
procedures  through  diagram  and  description.  The 
dissection  procedures  parallel  those  presented  in  the 
IVD  and  Video  materials.  The  lab  handout  instructed 
teams  to  locate  various  organs  during  the  dissection. 
In  teams  with  three  students,  one  student  selected  the 
job  of  surgeon,  another  acted  as  assistant  surgeon,  and 
a  director  relayed  instructions.  In  two-member  teams, 
the  assistant  surgeon  also  relayed  instructions. 

Criterion  Measures: 

The  30-item  achievement  test  was  composed 
of  several  parts  measuring  knowledge  of  frog  anatomy 
and  dissection  procedures.  Ptet  One  provided  three 
diagrams  on  which  students  were  directed  to  find  and 
label  ten  organs.  During  Part  Two,  students  went 
one  at  a  time  to  lab  stations  to  examine  four 
previously  dissected  frogs  marked  with  1 1  numbered 
pins.  Students  named  the  organs  marked  by  the  pins. 
In  Part  Three,  students  answered  nine  multiple  choice 
questions  (four  response  options  each)  about 
dissection  procedures.  Inter-item  consistency 
reliability  (KR20)  calculated  from  the  post-test 
administered  during  this  study  was  .58,  suggesting 


&2£  0* 


some  degree  of  heterogeneity  in  the  content  domain 
sampled. 

The  Attitudes  Toward  Dissection  measure 
was  constructed  of  20  items  with  a  four-point  Likert- 
type  response  scale.  Ten  of  the  items  were  adapted 
from  those  used  by  Strauss  &  Kinzie  (1992).  Ten 
additional  items  were  developed  specifically  for  this 
research.  Half  of  the  items  are  positively  phrased  and 
half  are  negatively  phrased,  as  described  by  Gable 
(1986).  Before  administration  of  this  measure,  it  was 
reviewed  and  critiqued  by  six  high  school  science 
teachers  and  six  educational  technologists,  and 
subsequently  revised.  Inter-item  consistency 
(Cronbach's  alpha)  of  this  instrument  was  calculated 
from  the  post-administration  in  this  study;  it  is 
estimated  to  be  .94.  The  Attitudes  Toward  Dissection 
measure  is  contained  in  the  Table  1. 


Insert  Table  1  about  here 


The  self -efficacy  measure  contains  23  items 
which  elicit  respondent  confidence  in  performing 
various  dissection  procedures  and  confidence  in 
identifying  portions  of  frog  anatomy.  Students 
indicated  their  level  of  agreement  on  a  four-point 
Likert  scale  to  items  beginning  with  "I  feel 
confident./1  and  ending  with  phrases  such  as 
"pinning  the  muscle  flaps  to  the  tray,"  and  "locating 
the  gall  bladder."  Alpha  reliability  of  this  measure 
was  calculated  from  student  response  to  the  post* 
administration  during  this  study  (rtf  *  .99),  and 
indicates  a  high  level  of  internal  consistency. 

During  the  dissection  laboratory,  six  student 
teams  worked  at  six  lab  benches,  three  on  each  side  of 
the  classroom.  Four  researchers  observed  and 
evaluated  dissection  performance.  Tic  researchers 
were  educators  well-trained  in  dissection  procedures. 
Two  researchers  observed  on  caw;  side  e£  the 
classroom  using  a  41  item  checklist  (Y/N)  to  evaluate 
team  performance.  Checklist  items  parallel  dissection 
procedures  outlined  in  the  lab  handout  and  preparatory 
materials.  The  checklist  contains  items  such  as, 
"Lifts  muscle  tissue  with  forceps?"  and  "Identify 
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kidneys?"  Teams  who  correctly  performed  a  step  in 
the  dissection  or  successfully  identified  a  specific 
organ  were  awarded  a  "Y"  rating;  those  that  did  not 
received  a  "N;"  all  Y  ratings  yielded  one  point  towards 
the  overall  evaluation  score.  In  the  event  that  a  team 
was  not  performing  a  step  properly  or  could  not 
identify  an  organ,  the  researcher  assisted  by  describing 
the  appropriate  technique  or  indicating  the  organ. 
Each  researcher  evaluated  one  team  alone,  and  one 
team  in  consort  with  another  researcher.  This  was  to 
ensure  reliability  of  the  evaluation.  In  all  cases,  joint 
observation  yielded  congruent  evaluation  ratings. 

Finally,  four  questions  soliciting  student 
preferences  for  various  study  activities  were  included 
at  the  end  of  the  affective  post-measures.  Students 
were  asked,  "If  you  were  given  a  choice  of  study 
activities  in  school,  which  would  you  prefer?" 
Students  selected  one  choice  from  each  of  the 
following  pairs:  1)  Studying  frog  dissection  and 
anatomy  or  studying  another  topic  within  biology;  2) 
Watching  a  videotape  or  film  on  dissection  or  using  a 
computer  program  on  dissection;  3)  Using  a  computer 
program  or  watching  a  videotape  or  film  on  dissection 
or  conducting  an  actual  dissection;  and  4)  Conducting 
a  dissection  after  using  a  computer  program  or  after 
watching  a  videotape  or  him  on  dissection  or 
conducting  a  dissection  without  using  any  prior 
instructional  materials. 

Design: 

This  study  employed  a  prc-tcst/post-test 
design.  One-way  Analysis  of  Variance  and 
Covariance  (ANOVA,  ANCOVA)  procedures  were 
employed,  except  where  noted  All  Analyses  were 
completed  with  SPSS  PC+  v.  3.0. 

RESULTS 

As  shown  in  Table  2,  all  subjects  evidenced 
significant  gains  in  achievement  test  performance 
between  the  pretest  (Af  »  10.05  out  of  30  possible, 
SD  =  3.96)  and  post-test  (Af  -  21 .24,  SD  -  4.76). 
Repeated  Measures  analysis  indicates  this  overall  gain 
to  be  significant,  F(l,  58)  ■  238.00,  /x.0001. 
Attitudes  toward  dissection  remained  relatively  stable 
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from  day  one  (M  -  50.89  out  of  80  possible,  SD  - 
11.42)  to  day  four  (M  .  52.50,  SD  -  12.21).  Self- 
efficacy  with  dissection  procedures,  on  the  other  hand, 
increased  from  the  administration  of  the  pit-measure 
(Af  -  64.95  out  of  100  possible,  SD  -  20.21)  to 
completion  of  the  post-measure  (M  -  72.73,  SD  - 
20.44),  F(l,  58)-  17.01,  p<.0001. 


Insert  Table  2  about  here 


In  the  analyses  that  follow,  premeasure 
scores  are  used  as  covariates  in  the  examination  of 
postmeasure  scores.  To  begin,  overall  analyses  were 
conducted  on  each  measure.  The  outcomes  indicated 
treatment  effects  for  Achievement,  (depicted  in  Table 
3)  and  Dissection  Performance  (shown  in  Table  4). 
No  treatment-related  differences  were  suggested  for 
Attitudes,  Self-Efficacy,  or  Dissection  Time.  Teams 
required  an  average  of  36.69  minutes  overall  (SD  • 
6.86)  for  the  dissection.  Planned  comparisons  were 
then  conducted  on  Achievement  and  Dissection 
Performance. 


IVD  Prep  subjects  were  compared  to  Diss  Only 
subjects  oaboth  dissection  performance  and 
achievement  Next,  to  determine  the  relative  effects 
of  interactive  practice  during  the  dissection 
preparation,  subjects  in  the  IVD  Prep  group  were 
compared  to  those  in  the  Video  Prep  group,  again 
examining  dissection  performance  and  achievement 
outcomes. 

Dissaciinn  ^fflrmilttfi  Subjects  who  used 
the  dissection  simulation  as  preparation  for  the  actual 
dissection  (IVD  Prep,  M  -  38.94  out  of  41  possible, 
SD  u  2.08)  were  found  to  be  more  effective  in 
conducting  the  dissection  than  those  who  had  received 
no  preparation  (Diss  Only,  M  -  32.21,  SD  -  4.53), 
F(l,28)-28.52,p<.0001.  The  dissection 
simulation  with  its  interactive  practice  activities  also 
proved  to  be  more  effective  than  than  linear  videotape 
'T  (Video  Prtp,  M   33.13,  SD  -  5.94)  in  preparing 
students  to  conduct  dissection,  F(l,29)  ■  13.54, 
jx.001.  Table  5  contains  the  results  of  these 
Dissection  Performance  analyses. 


Insert  Table  5  about  here 


Insert  Tables  3  &  4  about  here 


Simulation  versus  Dt^finn 

The  first  comparison  contrasted  subjects 
receiving  the  interactive  videodisc  based  simulation 
only  (IVD  only)  with  those  conducting  the  dissection 
only  (Diss  Only).  No  achievement  differences  were 
found  between  the  IVD  only  group  (M^  ]  0.00)  and 
the  Diss  Only  group  (A/l-M»7.87). 

interactive  simulation  »c  prr.nflratinn 

The  second  and  third  planned  comparisons 
were  directed  to  determine  the  relative  efficacy  of 
using  the  IVD-based  dissection  simulation  as  a 
preparation  for  students  who  will  go  on  to  conduct 
dissections.  To  examine  the  effects  of  the  interactive 
simulation  as  preparation  versus  no  preparation  at  all, 


Achievement  Students  given  the  IVD 
simulation  as  preparation  (IVD  Prep,  A/^-14.69) 
learned  more  than  students  not  given  any  preparation 
prior  to  conducting  a  dissection  (Diss  Only, 
WI-HI-7.87),F(1,28)-  15.24fAK.001.  No  difference 
in  achievement  was  found  between  IVD  Prep  subjects 
and  Video  Prep  subjects  (A/,^12.23).  See  Table  6 
for  the  statistical  summaries. 


Insert  Table  6  about  here 


The  Value  of  Dksectinn  flftfir  }  VP  Prfrpamriry 

In  the  final  planned  comparison,  students 
using  the  dissection  simulation  followed  by  actual 
dissection  0  VD  Prep)  were  compared  to  those  using 
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\  simulation  without  dissection  QVD  only).  The 
ent  was  to  deternine  the  additive  effects  that  actual 
section  might  have  following  use  of  the 
nulation.  Statistical  outcomes  (see  Table  6) 
ggcst  that  the  combination  of  the  interactive 
section  simulation  and  actual  dissection 
ffaU=14.69)  was  more  effective  in  promoting 
erall  achievement  than  the  simulation  alone 
^=10.00),  F(l,28) «  7.56,  p<.01. 

eferences  for  Study  Activities 

Multivariate  analysis  of  variance 
1ANOVA)  outcomes  indicated  no  significant 
aween-group  differences  at  the  multivariate  level  in 
bject  preferences  for  study  activities.  Subjects  were 
en  pooled  and  their  responses  analyzed  via  the  Chi 
luare  test  to  determine  if  their  expressed  preferences 
ere  significantly  different  from  those  expected  by 
iance(50%).  Subjects  reported  a  strong  preference 
r  dissection  when  preceded  by  some  form  of 
eparation  (computer  program,  video,  or  film)  (87%) 
/er  dissection  without  any  such  preparation  (13%), 
*(l,tf  =  60)  =  30.82,/k.0Q1.  Preferences  were 
so  expressed  for  use  of  a  computer  program  on 
issection  (68%)  over  a  related  video  of  film  (32%), 
2(l,tf  *60)  =  7.35,p<.01.  No  significant 
ifferences  were  noted  between  student  preferences  for 
se  of  a  dissection  substitute  (computer  program, 
ideo  or  film)  and  actual  dissection,  or  between  the 
udy  of  dissection  and  anatomy  and  another  topic 
ithin  biology. 

DISCUSSION 
achievement  and  Performance 

Several  interesting  achievement  outcomes 
rere  obtained  in  this  research.  First,  an  interactive 
issection  simulation  was  found  to  be  at  least  as 
ffective  as  actual  dissection  in  promoting  learning 
bout  frog  anatomy  and  dissection  procedures.  This 
utcome  supports  the  preliminary  results  reported  by 
trauss  &  Kinzie  (1992)  with  the  same  dissection 
imulation,  as  well  as  outcomes  reported  by  Prentice 
t  al.  (1977)  and  Welser  (1969).  At  the  least,  these 
esults  suggest  that  effective  alternatives  can  be 


offered  to  students  or  educators  who  desire  other 
approaches  to  learning  about  vertebrate  anatomy. 

A  second  finding  of  note  was  the 
effectiveness  of  the  interactive  dissection  simulation 
as  preparation  for  actual  dissection.  Students  using 
the  simulation  as  preparation  performed  more 
effectively  during  the  dissection  than  those  who  had 
received  no  preparation,  and  more  effectively  than 
students  viewing  a  linear  videotape  as  preparation. 
The  differential  effectiveness  of  student  teams  during 
actual  dissection  is  not  to  be  minimized-in  a  typical 
classroom  containing  20  to  30  students,  a  single 
instructor  is  hard-pressed  to  lend  assistance  to  all 
student  teams  conducting  a  dissection.  Iu  this  study 
the  four  researchers  present  during  the  dissection 
laboratory  were  busy  with  only  six  dissection  teams. 
Instructional  preparation  which  can  ensure  student 
effectiveness  in  upcoming  laboratory  situations  has 
indisputable  value. 

The  performance  advantage  for  the  I VD  Prep 
treatment  tended  to  translate  into  an  achievement 
benefit  as  well:  students  using  the  interactive 
simulation  as  preparation  for  the  dissection  learned 
more  overall  than  students  receiving  no  preparation. 
And  as  might  be  expected,  following  an  instructional 
activity  like  the  interactive  videodisc-based  dissection 
simulation  with  an  actual  dissection  led  to  greater 
overall  learning  than  use  of  the  simulation  alone. 

It  could  be  argued  that  the  apparent  advantage 
of  the  dissection  simulation  over  the  linear  videotape 
was  due  not  to  the  provision  of  interactive  practice 
but  rather  to  the  time  differential  between  the  two 
treatments:  IVD  Prep  subjects  spent  an  average  time 
of  39.4  minutes  with  the  simulation;  Video  Prep 
subjects  viewed  the  videotape  for  IS  minutes. 
However,  since  identical  video  materials  were 
displayed  in  both  treatments,  the  time  differential  was 
due  solely  to  the  interactive  practice  activities 
contained  in  the  dissection  simulation. 

AtfitndftsandSclf>Efficacv 

That  no  significant  changes  were  found  in 
attitudes  towards  dissection  is  not  surprising; 
theorists  have  long  contended  that  affective  values  are 
relatively  stable  and  are  modified  only  over  time 
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(Dick  A,  Carey,  1990;  Kerlinger,  1973).  Self-efficacy 
with  dissection  procedures  increased  for  all  groups  as 
a  result  of  their  activities,  but  no  between-group 
differences  were  significant 

Preferences  for  Sttriv  Arrivi^ 

Results  of  the  Chi  square  analyses  suggest 
the  value  that  students  perceive  in  appropriate 
preparation  for  dissection:  87%  of  the  participants  in 
this  study  indicated  a  preference  for  dissection  when 
preceded  by  a  computer  program,  videotape,  or  film 
on  dissection  while  only  13%  preferred  a  dissection 
without  preparation.  The  results  of  these  analyses 
also  point  to  the  attraction  that  computers  may  hold 
for  individuals  in  this  population-about  two-thirds  of 
the  students  expressed  a  preference  for  use  of  a 
computer-based  program  on  dissection  over  a  related 
video  or  film. 

Recommendations  for  Educational  ^tiftt 

As  mentioned  previously,  findings  reported 
here  suggest  the  efficacy  of  providing  a  dissection 
simulation  as  an  alternative  to  laboratory  dissection. 
For  those  educators  who  do  not  have  access  to 
interactive  videodisc  hardware  and  software,  other 
viable  dissection  alternatives  exist  (though  they  were 
not  tested  in  this  research):  use  of  less  sophisticated 
computer  programs,  activities  involving  anatomical 
models,  and  the  study  of  live  frogs  in  the  classroom 
(Hairston,  1990). 

However,  use  of  such  a  simulation  is  not  the 
equivalent  of  performing  a  laboratory  dissection: 
students  viewing  frog  specimens  on  a  computer 
screen  will  not  have  the  same  sensory  experience  as 
students  examining  actual  frog  tissues  and  organs. 
They  may  not  feel  the  same  sense  of  personal 
discovery.  For  these  reasons,  some  educators  may 
prefer  to  include  dissection  laboratories  in  their 
biology  classes.  Results  obtained  here  underline  the 
importance  of  preparing  students  for  the  dissection 
experience.  If  possible  this  instructional  preparation 
should  include  dissection-related  practice  activities.  In 
this  study,  students  who  received  interactive  practice 
performed  subsequent  dissections  much  more 
effectively  and  required  much  less  assistance  than 


those  who  viewed  a  linear  videotape  as  preparation  or 
those  receiving  no  preparation  at  all.  The  IVD 
preparation  group  also  learned  more  than  did  the 
group  conducting  a  dissection  with  no  preparation. 
S  tudents  across  groups  expressed  a  preference  for 
dissection  when  prepared  by  computer  program, 
videotape,  or  film  over  dissection  alone,  and  a 
preference  for  use  of  computer  programs  as 
preparation  over  videotape  or  films  for  the  same 
purpose. 

Finally,  following  up  an  activity  like  the 
dissection  simulation  with  actual  dissection  may  lead 
to  increased  learning  over  use  of  the  simulation  alone. 
It  should  be  noted,  however,  that  alternative  learning 
activities  besides  dissection  could  be  considered  as 
valuable  follow-up  activities.  It  is  conceivable  that 
the  achievement  benefit  found  for  the  simulation  plus 
dissection  could  also  be  found  for  the  simulation 
followed  by  any  number  of  other  activities  focusing 
on  frog  anatomy. 

Recommendations  fnr  ft^m  R^rrh 

Among  Jie  research  possibilities  in  this  area 
are  explorations  involving  the  various  dissection 
alternatives  that  exist  What  is  their  potential  relative 
to  laboratory  dissection  to  enhance  student  learning? 
Could  a  dissection  simulation  be  followed  up  by 
other  activities  besides  dissection  for  achievement 
gains  similar  to  those  found  here?  Through  research 
projects  such  as  these,  we  can  better  define  the 
attributes  of  instructional  activities  having  the  most 
educational  potential  for  biology  classrooms. 
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Tabic  1 

ATTITUDES  TOWARD  DISSECTION  McaSMB 


SURVEY:  HOW  DO  YOU  FEEL  ABOUT  DISSECTION? 


This  survey  has  20  statements  about  the  use  of  dissection  for  educational  purposes.  After  reading 
each  satement,  please  indicate  the  extent  to  which  you  agree  or  disagree,  by  circhngthe  number  to  the  right 
of  each  statement.  Please  respond  to  all  statements.  There  are  no  correct  or  incorrect  responses 


:rlc 


I 

Disagree 

i 

Disagree 

i 

Agree 

I 

Strongly 
Agree 

1. 

I  don't  see  how  frog  dissection  will  help  me  to 

kam  about  froa  anatomy  ] 

2 

3 

4 

2. 

AJlimals  can  he  treated  with  respecting  Action.  i 

2 

3 

4   

3. 

Pisseajonisanunoleasantacdvitv.  i 

2 

3 

4 

4. 

Biology  students  should  dissect  an  animal  to  help 
them  leam  about  an^omYl  j 

2 

3 

4 

5. 

l  am  distuibedbv  the  jdeaof  disscctinii  an  animal  1 

 *> 

A 
*t 

6. 

Dissectiop  makes  biology  mnm  interesting  1 

A 

7. 

Dissection  is  not  a  useful  way  to  learn  about  the 
Structure  and  funding  of  animals.  i 

2 

 J  

A 

8. 

It  is  morally  acceptable  for  man  to  harm  or  destroy 
animals  for  education  and  m^rrh  j 

2 

1 

A 

9. 

I  believe  dissection  is  an  effective  way  to  study 

the  anatomv  of  an  anim^,  j 

2 

 J        ,  ■ 

10. 

I  fed  comfortable  with  the  idea  of  conducting  a 
dissection  j 

i  „ 

 i  

— a   

11. 

To  help  me  learn  about  anatomy,  there  are  more 
Practical  activities  thai^  ffiyftction.  j 

m  „ 

2 

,JL  

12. 

Animals  should  not  be  harmed  for  the  purposes  of 
education  and  research  j 

u 

2 



3 

4 

13. 

Learning  about  frog  anatomy  through  dissection 
will  help  me  to  learn  about  the  anatomy  of  other 
onanisms.  j 

2 

3 

4 

14. 

I  do  not  think  that  learning  about  frog  dissection 

will  be  useful.  t 

2 

3 

4 

Dissecrinn  increases  qw  respect  fpr  animals  1 

2 

_  2 

4 

16. 

My  biology  class  would  be  more  enjoyable  without 
dissection.  j 

2 

3 

4 

17. 

I  feel  okay  about  dissecting  a  frog  in  order  to  learn 
about  froe  anatomv.  1 

2 

.  3 

4 

18. 

It  is  not  veiy  interesting  to  conduct  a  dissection 

.and  learn  about  frog  anatomv.  , 

2 

3 

4 

19. 

The  study  of  anatomy  does  not  justify  the  dissection 
OLibioloricalorsanism.  j 

2 

3 

4 

20. 

I  am  interested  in  finding  out  first-hand  about  frog 
anatomv  throuch  dissecting  ^ 

2 

.  3 

.,  4 

m 
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Tabic  2 

Mean  Achievement.  Attitude.  Self-Efficacy,  and  Performance  Scores 


Achievement        Attitudes  Self-Efficacy  Dissection 

Group  o  Ere  Easi  Etc  East  Etc  East  Ecrf  

Entire  Group  61 

M  10.05a  21.24  50.89b  52.50  64.95c  72.73  34,91d* 

SD  3.96  4.76  11.42  12.21  20.21  20.44  5.30 

IVD  Prep  16 

M  8.81  23.50  50.56  53.69  72.56  79.00  38.94 

SD  2.29  2.90  10.42  10.40  17.87  15.54  2.08 

Video  Prep  15 

M  10.33  22.46  49.60  53.14  61.87  74.36  33.13 

SD  "22  4.88  12.60  11.22  18.64  18.27  5,94 

Diss  Only  15 

M  10.73  18.60  52.93  53.67  60.87  69.27  32.21 

SD  4.13  5.53  9.84  14.08  21.46  24.66  4.53 


IVD  Only  15 

M  10.40    20.40        50.47    49.47        63.93  68.00 

SD  3.83     4.32        13.42    13.55        22.63  22.37 


*  Total  Score,  out  of  30  possible. 

b  Total  Score,  out  of  80  possible.  Higher  Scores  indicate  more  positive  attitudes  towards 
dissection. 

c  Total  Score,  out  of  100  possible.  Higher  Scores  indicate  greater  levels  of  perceived  self  -efficacy 

with  dissection  procedures 
d  Total  Score,  out  of  41  possible. 

e  Mean  Dissection  Performance  Score  for  the  Entire  Group  excludes  IVD  Only  subjects,  who  did 
not  conduct  the  dissection. 


Tabk3 

ANCQVA  OmcnmtIS  for  rtll  Treatment  fimnps  nn  AnhiPvrn1PTlt 
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Source 

SS 

df 

MS 

F 

Pretest» 

46.78 

1 

46.78 

2.50 

Treatment 

258.88 

3 

86.29 

4.61** 

Error 

1011.02 

54 

18.72 

Prt-measures  were  employed  as  covariates  in  all  ANCOVA  procedures 
p  <  .01 


Table  4 

ANQVA  Outcomes  fnra]|  Groups  nn  Dissection  P^rmamX* 


Source 

SS 

df 

MS 

F 

Treatment 
Error 

408.62 
825.03 

2 

42 

204.31 
19.64 

10.40*** 

Includes  only  those  groups  that  conducted  a  dissection. 
p  <  .001 
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Table  5 

ANOVA  Outcomes  for  Planned  Comparisons  on  Dissection  Performance 

IVD  Prep  vs.  Diss  Only 

Source  SS  df  MS  F 

Treatment  337.50  1  337.50  28.52**** 

Error  331.30  28  11.83 


TVDPrepvs.  Video  Prep 


Source  SS  df  MS  F 

Treatment  260.81  1  260.81  13.54*** 

Error  558.67  29  19.26 


***  p<.001 
****     p  <  .0001 
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Table  6 

ANCQVA  OllTCOmes  for  Planned  Company  on  Achiwmont 


Interactive  Dissection  Simulation 

IS 


IVD  Prrp  vs,  Diss  Only 


Source  SS  df  MS  F 


Pretest* 

.  28.35 

1 

28.35 

1.72 

Treatment 

250.64 

1 

250.64 

15.24*** 

Error 

460.49 

28 

16.45 

Source  SS  df  MS  F 

198  1  1.98  0.13 

9-53  1  9.53  0.61 

407.46  26  15.67 


IVDPrenvs.  TVnf)n|y 

Source                 SS  df  MS  F 

11.08  1  n.08  0.87 

95.83  1  95.83  7.56** 

355.09  28  12.68 


Pretest 

Treatment 

Error 


Pretest 

Treatment 

Error 


*•  ^  Q5hievcmcnt  Pretest  85  covariate  in  these  ANCOVA  procedures. 
***  p  <  .001 
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Effects  of  Cooperative  Learning  and  Incentive 
on  Motivation  and  Performance 

A  number  of  studies  have  been  conducted  to  compare 
cooperative  and  individual  learning  strategies. 
According  to    Johnson  &  Johnson  (1989),  a  cooperative 
learning  strategy  allows  students  to  work  together  to 
increase  performance  and  achieve  shared  goals;  an 
individual  learning  strategy  requires  students  to  work 
by  themselves  to  accomplish  their  own  goals,  several 
reviews  of  research  suggest  that  cooperative  learning 
affects  student  performance,  productivity,  transfer  of 
learning,  time  on  task,  and  attitude     (Johnson  & 
Johnson,  1989;  Rysavy  &  Sales,  1991;  Sharan,  1980; 
Slavm,  1990). 

The  results  of  cooperative  learning  research  has 
inspired  instructional  technologists  to  examine  how 
cooperative  groups  can  be  implemented  with  media 
originally  designed  for  individual  learning.  Results 
of  studies  conducted  to  investigate  cooperative 
learning  and  computer-assisted  instruction  are  mixed. 
Some  researchers  report  that  cooperative  learning 
positively  affected  performance  in  CAI  lessons  (Dalton, 
Hannafin,  &  Hooper,  1989;  Johnson,  Johnson,  &  Stanne, 
1985) ,  while  others  have  not  found  a  significant  effect 
for  performance  when  learners  used  cooperative  CAI 
(Carrier  &  Sales,  1987).    Research  on  cooperative 
learning  and  instructional  television  suggests  that 
students  working  in  groups  are  more  motivated  than 
those  who  work  alone,  but  performance  in  these  settings 
is  influenced  by  one's  affiliation  motives  (Klein  & 
Pridemore,  1992). 

The  mixed  results  in  studies  that  have  examined 
cooperative  learning  with  media  may  be  due  to  the 
rewards  provided  to  students.     Some  scholars  indicate 
that  providing  rewards  to  students  who  work  in 
cooperative  groups  can  negatively  impact  their 
achievement,  interest,  and  continuing  motivation  (Kohn, 
1991) .     However,  many  researchers  suggest  that 
providing  rewards  to  students  is  an  important  component 
in  cooperative  learning  (Johnson  &  Johnson,  1989; 
Sharan,  1980;  Slavin;  1990). 

Researchers  have  distinguished  between  cooperative  and 
individualistic  reward  structures  (Deutsch,  1949; 
Johnson  &  Johnson,  1974;  Michaels,  1977;  Slavin,  1977). 
A  cooperative  reward  structure  provides  all  members  of * 
a  group  the  same  reward  based  on  the  performance  of  the 
whole  group.    An  individualistic  reward  structure 
provides  each  individual  with  a  reward  based  on  their 
own  performance. 
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In  cooperative  learning  settings,  the  effect  of  these 
types  of  rewards  on  performance  is  unclear.  Slavin 
(1991),  suggests  "...  almost  every  study  of 
cooperative  learning  in  which  the  cooperative  classes 
achieved  more  than  the  traditional  control  groups  used 
some  sort  of  group  reward"  (p.  89) .    However,  other 
researchers  have  not  found  cooperative  rewards  to  be 
superior  to  individualistic  rewards  when  performance  is 
measured  (Hamblin,  Hathaway,  &  Wodarski,  1971; 
Michaels,  1977;  Niehoff  &  Mesch,  1991).     These  results 
may  be  due  to  the  type  of  individualistic  reward 
offered  to  students  who  work  in  cooperative  groups. 

The  purpose  of  the  current  study  was  to  examine  the 
effect  of  cooperative  learning  and  type  of  reward  on 
performance  and  continuing  motivation.    The  study  is  a 
continuation  of  a  program  of  research  designed  to 
investigate  how  cooperative  groups  can  be  implemented 
with  media  originally  designed  for  individual  learning. 

Method 

Subjects  were  126  undergraduate  education  majors  (30 
males,  96  females)  in  the  first  semester  of  a  teacher 
training  program  at  a  large  southwestern  university. 
All  subjects  were  enrolled  in  a  required  course  in 
educational  psychology;  participation  in  the  study 
fulfilled  a  requirement  for  this  course. 

Materials  used  in  this  study  were  an  instructional 
television  lesson,  a  posttest,  and  a  continuing 
motivation  survey.  The  instructional  television  lesson 
was  from  the  series  Instructional  Theory:  A  nine  unit 
mini-course  (Gerlach,  1973).    The  lesson  included  a 
videotape  and  a  workbook  that  provided  instruction  on 
the  topic  of  objectives-based  assessment.  The 
videotape  was  approximately  30  minutes  in  length.  It 
was  divided  into  seven  segments  which  presented 
information  and  examples  on  the  content  of  the  lesson. 
After  each  segment,  the  videotape  instructed  subjects 
to  turn  to  their  workbook  for  practice  and  feedback  on 
the  content  presented  in  that  segment.    Performance  was 
measured  using  a  15- item,  constructed  response 
posttest.    The  items  were  developed  to  evaluate  student 
mastery  of  the  instructional  objectives  for  the  lesson 
on  objectives-based  assessment.    The  maximum  score  on 
the  posttest  was  twenty  points.     Individual  answers 
were  checked  against  a  scoring  key  and  points  were 
assigned  for  each  answer.    Partial  credit  was  given  for 
questions  that  required  a  multiple  response.  One 
person  scored  all  of  the  items  on  this  test.  The 
Kuder-Richardson  internal-consistency  reliability  of 
the  posttest  ranges  from  .69  to  .81  (Klein  &  Pridemore, 
1992).    Continuing  motivation  was  assessed  using  a 
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paper  and  pencil  survey.    This  survey  consisted  of 
seven  items  that  measured  the  degree  to  which  a  subject 
would  want  to  return  to  tasks  like  those  used  in  the 
study.    The  Cronbach  alpha  internal-consistency 
reliability  estimate  of  this  survey  was  .60. 

Subjects  were  randomly  assigned  to  one  of  six  treatment 
conditions.    All  possible  combinations  of  instructional 
method  (individual  versus  cooperative)  and  type  of 
reward  (task,  performance,  none)  were  equally 
represented  after  assignment  to  groups.  Additional 
random  assignment  was  conducted  for  the  cooperative 
learning  treatments;  subjects  in  these  conditions  were 
randomly  assigned  to  groups  of  three  (triads) . 

At  the  beginning  of  the  lesson,  subjects  in  all 
treatments  were  informed  that  they  would  be  viewing  an 
instructional  television  program  on  objectives-based 
assessment  and  that  they  would  be  using  a  workbook  to 
receive  practice  and  feedback  on  the  content  of  the 
lesson.     Subjects  were  told  to  write  the  answer  to  each 
practice  exercise  in  the  workbook  and  read  the  feedback 
that  followed  each  exercise.     In. addition,  subjects  in 
all  groups  were  directed  to  read  the  first  two  pages  of 
the  workbook  to  receive  the  lesson  objectives  and  were 
told  that  they  would  each  be  completing  a  test  on  the 
content  of  the  lesson.     Other  procedures  and  directions 
were  different  depending  on  treatment  condition. 
Subjects  received  specific  directions  for  implementing 
either  an  individual  or  a  cooperative  learning 
strategy.     Subjects  working  alone  were  each  given  a 
workbook,  instructed  to  work  independently  during  the 
lesson,  and  told  to  do  their  best  work.     Subjects  in 
the  cooperative  learning  conditions  were  randomly 
assigned  to  a  triad.     Each  triad  was  given  a  workbook 
and  told  to  (a)  work  together  during  the  lessen,  (b) 
discuss  all  practice  exercises  and  any  disagreements 
over  the  answers,  and  (c)  discuss  the  given  feedback. 

Subjects  also  received  specific  directions  concerning 
the  reward  for  learning  depending  on  their  treatment 
group.     Subjects  in  the  task  reward  groups  were  told, 
"Your  participation  in  this  lesson  is  worth  ten  points 
toward  your  course  grade."    Subjects  in  the  performance 
reward  groups  were  told,  "Your  participation  in  this 
lesson  can  be  worth  as  much  as  ten  points,  but  the 
number  of  points  you  earn  depends  on  how  much  you  learn 
from  the  lesson."    Finally,  subjects  in  the  no  reward 
groups  were  told,  "Your  participation  in  this  lesson 
will  help  you  to  be  successful  both  on  the  final  exam 
and  on  the  course  project." 
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After  the  above  instructions  were  provided,  the 
videotape  was  started  for  each  treatment  condition. 
When  Segment  1  was  completed,  the  tape  was  stopped  and 
subjects  did  Exercise  1.    When  Exercise  1  was 
completed,  the  videotape  was  started  again.     This  cycle 
was  continued  until  all  seven  sections  of  the  lesson 
were  finished.    Upon  completion  of  these  activities, 
all  workbooks  were  collected  and  each  subject 
individually  completed  the  continuing  motivation 
survey.     One  week  later,  £  11  subjects  were  given  the 
posttest  and  were  required  to  work  individually  to 
complete  it.    One  week  after  the  posttest  session, 
subjects  were  informed  that  each  of  them  would  earn  ten 
points  toward  their  course  grade  regardless  of  how  well 
they  did  on  the  posttest. 

Results 

Analysis  of  variance  (ANOVA)  revealed  that  type  of 
instructional  method  had  a  significant  effect  on 
performance,  F(l,  119)  =  4.07,  p  <  .05,  H§£  =  13.76,  £S 
=  .34.    Subjects  who  worked  alone  performed  better  on 
the  posttest  (M  -  11.9,  SD  =  3.7)  than  those  who  worked 
cooperatively  (M  =  10.6,  SD  =  3.6).    Type  of  reward  did 
not  have  a  significant  effect  on  performance. 
Furthermore,  a  significant  interaction  between 
instructional  method  and  type  of  reward  was  not  found. 

Multiple  analysis  of  variance  (MANOVA)  suggested  that 
type  of  instructional  method  had  a  significant  effect 
on  continuing  motivation,  E(7,  113)  -  4.10,  p  <  .001. 
Type  of  reward  did  not  affect  continuing  motivation  and 
a  significant  interaction  between  instructional  method 
and  type  of  reward  was  not  found.    Univariate  analyses 
revealed  that  subjects  who  worked  alone  expressed  more 
continuing  motivation  than  those  who  worked 
cooperatively  for  instructional  television  programs 
that  require  individual  work,  E(l,119)  =  16.23,  p  < 
.001,  HS£  =  1.34,  £S  =  .69,  and  for  other  activities 
that  require  individual  work,  £(1,119)  =  7.50,  p  <  .01, 
MSe  «=  1.44,  ££«  .48.     In  addition,  subjects  who  worked 
cooperatively  expressed  more  continuing  motivation  than 
those  who  worked  alone  for  activities  that  require 
working  with  other  students,  £(1,119)  =  5.82,  p  <  '05» 
MSe  =  .94,  £S  =  .42. 

Discussion 

The  purpose  of  this  study  was  to  explore  the  effect  of 
cooperative  learning  and  type  of  reward  on  performance 
and  continuing  motivation.     Subjects  used  either  a 
cooperative  or  individual  learning  strategy  while 
receiving  information,  examples,  practice,  and  feedback 
from  an  instructional  television  lesson. 
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Results  indicated  that  subjects  who  worked  alone 
performed  better  and  expressed  more  continuing 
motivation  than  those  who  worked  cooperatively.  These 
findings  lend  support  to  other  studies  which  suggest 
that  a  cooperative  strategy  may  not  affect  educational 
outcomes  in  some  settings.    While  cooperative  learning 
has  influenced  student  performance  and  attitudes  in 
classroom  settings  (Johnson  &  Johnson,  1989;  Sharan, 
1980;  Slavin,  1990),  cooperative  learning  has  not 
always  influenced  performance  when  it  is  implemented 
with  media  originally  designed  for  individual  learning 
(Carrier  &  Sales,  1987;  Klein  &  Pridemore,  1992). 

The  results  of  the  current  study  may  have  occurred 
because  of  how  students  approach  learning  from 
instructional  television  lessons.     In  our  society, 
television  viewing  is  an  individual  experience  with 
little  opportunity  for  interaction  among  viewers. 
Furthermore,  television  is  typically  implemented  for 
individual  use  in  instruction.    Most  students  have  very 
little  practice  working  with  other  people  when  using 
instructional  television  lessons. 

Even  though  subjects  in  the  triads  were  given  specific 
directions  for  implementing  a  cooperative  strategy 
during  the  lesson,  informal  observations  suggest  that 
many  groups  did  not  follow  these  directions.  After 
each  segment  of  the  tape  was  stopped,  subjects  in 
several  of  the  triads  quietly  read  the  question  in  the 
workbook  to  themselves.    One  of  the  group  members  would 
usually  ask  the  others  for  the  answer  and  would  write 
it  in  the  workbook.    Very  little  on-task  discussion 
occurred  in  these  groups.    Groups  that  finished  the 
practice  before  other  triads  in  the  room  would  usually 
talk  about  topics  that  were  unrelated  to  the  lesson. 
These  behaviors  are  somewhat  different  than  those  of 
the  subjects  who  worked  alone.    After  reading  the  each 
question  and  answering  it,  many  individuals  who 
finished  the  practice  before  others  in  the  room  usually 
reviewed  the  item  or  looked  ahead  to  the  next  item  in 
their  workbook.     This  additional  on-task  behavior  may 
have  enhanced  the  performance  of  subjects  who  worked  by 
themselves. 

Another  explanation  for  the  results  found  in  this  study 
may  be  due  to  the  nature  of  the  reward  structure 
provided.    Subjects  in  both  the  cooperative  and 
individual  learning  conditions  were  placed  in  one  of 
three  individual  reward  structures  (i.e.:  task, 
performance,  none) .     Even  subjects  in  the  no  reward 
conditions  had  some  incentive  for  individual  learning, 
because  they  were  told  that  the  lesson  would  increase 
success  in  the  course.     None  of  the  subjects  in  the 
cooperative  groups  were  placed  in  a  cooperative  reward 
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structure.    Slavin's  (1991),  suggestion  that  group 
rewards  have  a  strong  impact  on  cooperative  learning  in 
classroom  settings  may  also  be  legitimate  when 
cooperative  groups  use  media  that  were  originally 
designed  for  individual  learning. 

Future  research  should  continue  to  explore  the  use  of 
cooperative  learning  with  technologies  that  were 
originally  developed  for  individual  learning.  Studies 
should  investigate  how  different  reward  structures 
(e.g.,  individual,  cooperative,  competitive)  influence 
outcomes  in  educational  technology  settings.  Research 
should  also  be  conducted  to  examine  how  students  can  be 
taught  to  cooperate  together  when  they  use  media. 
These  studies  should  examine  the  quantity  and  quality 
of  group  interaction  when  cooperative  strategies  are 
implemented  with  media.     These  suggestions  will  assist 
us  in  determining  the  appropriate  use  of  cooperative 
learning. 
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Abstract 

The  purpose  of  this  study  was  to  investigate  the  effect  of 
cooperative  learning  and  type  of  reward  on  performance  and 
continuing  motivation.     Subjects  used  either  a  cooperative  or 
individual  learning  strategy  while  receiving  information, 
examples,  practice,  and  feedback  from  an  instructional  television 
lesson,     subjects  were  also  provided  with  one  of  three  rewards 
(i.e.:  task,  performance,  none).     Results  indicated  that  subjects 
who  worked  alone  performed  better  on  the  posttest  and  expressed 
more  continuing  motivation  than  those  who  worked  cooperatively. 
Type  of  reward  did  not  affect  performance  or  continuing 
motivation.     Implications  for  employing  cooperative  groups  in 
settings  that  were  originally  designed  for  individual  learning 
are  provided. 
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Effects  of  Cooperative  Learning  and  Type  of  Reward 
on  Performance  and  Continuing  Motivation 
A  number  of  studies  have  been  conducted  to  compare 
cooperative  and  individual  learning  strategies.    According  to 
Johnson  &  Johnson  (1989),  a  cooperative  learning  strategy  allows 
students  to  work  together  to  increase  performance  and  achieve 
shared  goals;    an  individual  learning  strategy  requires  students 
to  work  by  themselves  to  accomplish  their  own  goals.  Several 
reviews  of  research  suggest  that  cooperative  learning  affects 
student  performance,  productivity,  transfer  of  learning,  time  on 
task,  and  attitude     (Johnson  &  Johnson,  1989;  Rysavy  &  Sales, 
1991;  Sharan,  1980;  Slavin,  1990). 

The  results  of  cooperative  learning  research  has  inspired 
instructional  technologists  to  examine  how  cooperative  groups  can 
be  implemented  with  media  originally  designed  for  individual 
learning.     Results  of  studies  conducted  to  investigate 
cooperative  learning  and  computer-assisted  instruction  are  mixed. 
Some  researchers  report  that  cooperative  learning  positively 
affected  performance  in  CAI  lessons  (Dalton,  Hannafin,  &  Hooper, 
1989;  Johnson,  Johnson,  &  Stanne,  1985),  while  others  have  not 
found  a  significant  effect  for  performance  when  learners  used 
cooperative  CAI  (Carrier  &  Sales,  1987).    Research  on  cooperative 
learning  and  instructional  television  suggests  that  students 
working  in  groups  are  more  motivated  than  those  who  work  alone, 
but  performance  in  these  settings  is  influenced  by  one's 
affiliation  motives  (Klein  &  Pridemore,  1992). 
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The  mixed  results  in  studies  that  have  examined  cooperative 
learning  with  media  may  be  due  to  the  rewards  provided  to 
students.     Some  scholars  indicate  that  providing  rewards  to 
students  who  work  in  cooperative  groups  can  negatively  impact 
their  achievement,   interest,  and  continuing  motivation  (Kohn, 
1991) .     However,  many  researchers  suggest  that  providing  rewards 
to  students  is  an  important  component  in  cooperative  learning 
(Johnson  &  Johnson,   1989;  Sharan,   1980;  Slavin;  1990). 

Researchers  have  distinguished  between  cooperative  and 
individualistic  reward  structures  (Deutsch,  1949;  Johnson  & 
Johnson,  1974;  Michaels,   1977;  Slavin,  1977).     A  cooperative 
reward  structure  provides  all  members  of  a  group  the  same  reward 
based  on  the  performance  of  the  whole  group.    An  individualistic 
reward  structure  provides  each  individual  with  a  reward  based  on 
their  own  performance. 

In  cooperative  learning  settings,  the  effect  of  these  types 
of  rewards  on  performance  is  unclear.     Slavin  (1991)  ,  suggests 
" .   .   .  almost  every  study  of  cooperative  learning  in  which  the 
cooperative  classes  achieved  more  than  the  traditional  control 
groups  used  some  sort  of  group  reward"  (p.  89).     However,  other 
researchers  have  not  found  cooperative  rewards  to  be  superior  to 
individualistic  rewards  when  performance  is  measured  (Hamblin, 
Hathaway,  &  Wodarski,   1971;  Michaels,   1977;  Niehoff  &  Mesch, 
1991).    These  results  may  be  due  to  the  type  of  individualistic 
reward  offered  to  students  who  work  in  cooperative  groups. 
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The  purpose  of  the  current  study  was  to  examine  the  effect 
oi:  cooperative  learning  and  type  of  reward  on  performance  and 
continuing  motivation.     The  study  is  a  continuation  of  a  program 
of  research  designed  to  investigate  how  cooperative  groups  can  be 
implemented  with  media  originally  designed  for  individual 
learning.     While  several  studies  have  been  conducted  to  examine 
cooperative  learning  and  computers,  little  work  has  been 
conducted  to  examine  cooperative  learning  and  instructional 
television.     Adams,  Carson,  and  Hamm  (199  0)   suggest  that 
cooperative  learning  can  influence  attention,  motivation,  and 
achievement  when  students  use  the  medium  of  television.  These 
authors  indicate  that  "cooperative  strategies  which  engage 
students  in  examining,  comparing,  clarifying  and  evaluating 
enhance  individual  experiences"  (Adams,  Carson,  &  Hamm,   1990,  p. 
39)  . 

In  this  study,  subjects  used  either  a  cooperative  or 
individual  learning  strategy  while  receiving  instruction  from  a 
television  lesson.     The  independent  variables  were  instructional 
method  (individual  versus  cooperative)  and  type  of  reward  (task, 
performance,  none) .     The  dependent  variables  were  performance  and 
continuing  motivation. 

Method 

Subjects 

Subjects  were  126  undergraduate  education  majors  (3  0  males, 
96  females)   in  the  first  semester  of  a  teacher  training  program 
at  a  large  southwestern  university.    All  subjects  were  enrolled 
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ii\  i  required  course  in  educational  psychology;  participation  in 

the  study  fulfilled  a  requirement  for  this  course. 

Materials 

Materials  used  in  this  study  were  an  instructional 
television  lesson,  a  posttest,  and  a  continuing  motivation 
survey. 

The  instructional  television  lesson  was  from  the  series 
Instructional  Theory;  A  nine  nnit  minj^gfluxse  (Geriach,  1973). 
The  lesson  included  a  videotape  and  a  workbook  that  provided 
instruction  on  the  topic  of  objectives-based  assessment.  The 
videotape  was  approximately  30  minutes  in  length.     It  was  divided 
into  seven  segments  which  presented  information  and  examples  on 
the  content  of  the  lesson.     After  each  segment,  the  videotape 
instructed  subjects  to  turn  to  their  workbook  for  practice  and 
feedback  on  the  content  presented  in  that  segment.     For  example, 
Segment  4  provided  instruction  on  the  use  of  paper-and-pencil 
tests,  interviews,  and  observations  of  student  performance  or 
product.     After  providing  information  and  examples  of  these  three 
types  of  objectives-based  assessment,  the  tape  presented  viewers 
with  three  instructors  who  wished  to  evaluate  a  student's  work  of 
sculpture.     The  videotape  directed  subjects  to  "Turn  to  Exercise 
4  in  your  workbook"  where  they  were  asked  to  "Describe  the  best 
type  of  objectives-based  assessment  for  this  situation."  The 
workbook  then  provided  written  feedback  to  this  practice  item  on 
the  following  page. 
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Perfonnance  was  measured  using  a  15-item,  constructed 
response  posttest.     The  items  were  developed  to  evaluate  student 
mastery  of  the  instructional  objectives  for  the  lesson  on 
objectives-based  assessment.     The  following  is  an  example  of  an 

item  on  the  posttest: 

An  industrial  technology  instructor  is  teaching  his  students 
to  adjust  a  communications  receiver  so  that  the  sound  is  of 
certain  minimum  quality,  at  the  very  least.     How  should  he 
test  for  the  attainment  of  this  objective? 
The  maximum  score  on  the  posttest  was  twenty  points.  Individual 
answers  were  checked  against  a  scoring  key  and  points  were 
assigned  for  each  answer.     Partial  credit  was  given  for  questions 
that  required  a  multiple  response  such  as  "List  three  types  of 
objectives-based  assessment."    One  person    cored  all  of  the  items 
on  this  test.        The  Kuder-Richardson  internal-consistency 
reliability  of  the  posttest  ranges  from  .69  to  .81  (Klein  & 

Pridemore,  1992). 

Continuing  motivation  was  assessed  using  a  paper  and  pencil 
survey.     This  survey  consisted  of  seven  items  that  measured  the 
degree  to  which  a  subject  would  want  to  return  to  tasks  like 
those  used  in  the  study.    The  directions  on  this  survey 
indicated, 

Some  of  the  things  that  you  will  be  learning  in  the  teacher 
training  program  will  be  taught  using  activities  similar  to 
those  used  in  the  lesson  on  objectives-based  assessment. 
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We  are  interested  in  the  kinds  of  activities  that  you  would 
like  to  participate  in  as  you  continue  in  the  program. 

A  five-point  Likert  scale  was  used  1-n  ir**A~^+. 

ie  was  used  to  indicate  agreement  with  the 

statements: 

(1)  I  would  like  to  learn  more  about  objectives-based 
assessment; 

(2)  I  would  like  to  learn  more  about  objectives-based 
assessment  using  instructional  television; 

(3)  I  would  like  to  learn  more  about  other  topics 
using  instructional  television; 

(4)  I  would  like  to  use  instructional  television  programs 
that  require  me  to  work  with  other  students; 

(5)  I  would  like  to  use  instructional  television  program 
that  require  me  to  work  by  myself; 

(6)  1  would  like  to  particinato  in  ^.  . 

Hotuicipate  in  other  activities  that 

require  me  to  work  with  other  students; 

(7)  I  would  like  to  participate  in  other  activities  that 
require  me  to  work  alone. 

The  crotch  alpha  internal-consistency  reliability  estimate  Qf 

this  survey  was  .60. 

irxocegyriag 

subjects  were  randomly  assigned  to  one  o£  ^ 
conditions.     A11  possible  conMnations  Qf  instruotional  ^ 
(individual  versus  cooperative,  and  type  of  reward  (task 
performance,  none,  were  egually  represented  after  assignment  to 
groups.     Additional  random  assignment  was  conducted  for  the 
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cooperative  learning  treatments;  subjects  in  these  conditions 
w:*re  randomly  assigned  to  groups  of  three  (triads)  . 

Each  treatment  condition  was  implemented  in  a  separate  room; 
each  room  had  more  than  one  individual  or  cooperative  group 
present  at  a  time.     At  the  beginning  of  the  lesson,  subjects  in 
all  treatments  were  informed  that  they  would  be  viewing  an 
instructional  television  program  on  objectives-based  assessment 
and  that  they  would  be  using  a  workbook  to  receive  practice  and 
feedback  on  the  content  of  the  lesson.     Subjects  were  told  to 
write  the  answer  to  each  practice  exercise  in  the  workbook  and 
read  the  feedback  that  followed  each  exercise.     In  addition, 
subjects  in  all  groups  were  directed  to  read  the  first  two  pages 
of  the  workbook  to  receive  the  lesson  objectives  and  were  told 
that  they  would  each  be  completing  a  test  on  the  content  of  the 
lesson.     Other  procedures  and  directions  were  different  depending 
on  treatment  condition. 

Subjects  received  specific  directions  for  implementing 
either  an  individual  or  a  cooperative  learning  strategy. 
Subjects  working  alone  were  each  given  a  workbook,  instructed  to 
work  independently  during  the  lesson,  and  told  to  do  their  best 
work.     Subjects  in  the  cooperative  learning  conditions  were 
randomly  assigned  to  a  triad.     Each  triad  was  given  a  workbook 
and  told  to  (a)  work  together  during  the  lesson,   (b)  discuss  all 
practice  exercises  and  any  disagreements  over  the  answers,  and 
(c)  discuss  the  given  feedback. 


551 


0  -J  J 


Cooperative  Learning  &  Rewards 

10 

Subjects  also  received  specific  directions  concerning  the 
reward  for  learning  depending  on  their  treatment  group.  Subjects 
in  the  task  reward  groups  were  told,     "Your  participation  in  this 
lesson  is  worth  ten  points  toward  your  course  grade."  subjects 
in  the  performance  reward  groups  were  told,  "Your  participation 
in  this  lesson  can  be  worth  as  much  as  ten  points,  but  the  number 
of  points  you  earn  depends  on  how  much  you  learn  from  the 
lesson."    Finally,  subjects  in  the  no  reward  groups  were  told, 
"Your  participation  in  this  lesson  will  help  you  to  be  successful 

both  on  the  final  exam  and  on  the  course  project." 

After  the  above  instructions  were  provided,  the  videotape 

was  started  for  each  treatment  condition.    When  Segment  1  was 

completed,  the  tape  was  stopped  and  subjects  did  Exercise  1. 

When  Exercise  1  was  completed,  the  videotape  was  started  again. 

This  cycle  was  continued  until  all  seven  sections  of  the  lesson 

were  finished. 

Upon  completion  of  these  activities,  all  workbooks  were 
collected  and  each  subject  individually  completed  the  continuing 
motivation  survey,     one  week  later,  all  subjects  were  given  the 
posttest  and  were  required  to  work  individually  to  complete  it. 
One  week  after  the  posttest  session,  subjects  attended  a 
debriefing  session  where  they  were  informed  that  each  of  them 
would  earn  ten  points  toward  their  course  grade  regardless  of  how 
well  they  performed  on  the  lesson  posttest. 
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Design  and  Data  Analysis 

A  2  X  3  factorial  design  was  used,  with  instructional  method 
(individual  versus  cooperative)  and  type  of  reward  (task, 
performance,  none)  as  the  independent  variables.     The  dependent 
variables  were  performance  and  continuing  motivation.  Analysis 
of  variance  (ANOVA)  was  used  to  test  for  a  difference  between 
groups  on  the  posttest.     Multiple  analysis  of  variance  (MANOVA) 
was  used  to  test  for  an  overall  difference  between  groups  on  the 
continuing  motivation  survey.     This  analysis  was  followed  by 
univariate  analyses  on  the  individual  survey  items.     Alpha  was 
set  at  .05  for  all  statistical  tests.     Effect  size  estimates  (££) 
expressed  as  a  function  of  the  overall  standard  deviation  were 
also  calculated.     Data  for  125  out  of  the  126  subjects  were 
included  in  the  analyses,  since  data  for  one  subject  was 
incomplete. 

Results 

Performance 

Performance  was  measured  using  the  15-item,  constructed 
response  posttest.     Mean  scores  and  standard  deviations  for  the 
posttest  can  be  found  in  Table  1. 

Table  1  about  here 


Analysis  of  variance  (ANOVA)  revealed  that  type  of 
instructional  method  had  a  significant  effect  on  performance, 
£(1,   119)   =  4.07,  E  <   -05,  MS£  ■  13.76,  ££  =  .34.     Subjects  Who 
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worked  alone  performed  better  on  the  posttest  (M  «  11.9,  SD  = 
3.7)  than  those  who  worked  cooperatively  (M  -  10.6,  SD  =  3.6). 
Type  of  reward  did  not  have  a  significant  effect  on  performance. 
Furthermore,  a  significant  interaction  between  instructional 
method  and  type  of  reward  was  not  found. 
Continuing  Motivation 

Continuing  motivation  was  measured  using  the  seven-item 
scale.     Mean  scores  and  standard  deviations  for  individual 
continuing  motivation  items  can  be  found  in  Table  2. 


Table  2  about  here 


Multiple  analysis  of  variance  (MANOVA)  suggested  that  type 
of  instructional  method  had  a  significant  effect  on  continuing 
motivation,  £(7,   113)  =  4.10,  E  <  .001.     Type  of  reward  did  not 
affect  continuing  motivation  and  a  significant  interaction 
between  instructional  method  and  type  of  reward  was  not  found. 

Univariate  analyses  revealed  that  subjects  who  worked  alone 
expressed  more  continuing  motivation  than  those  who  worked 
cooperatively  for  instructional  television  programs  that  require 
individual  work,  Z(l,119)  =  16.23,  p  <  .001,  ES&  =  1.34,  ££  = 
.69,  and  for  other  activities  that  require  individual  work, 
1(1,119)  =  7.50,  E  <  .01,  Else.  =  1.44,        =  .48.     in  addition, 
subjects  who  worked  cooperatively  expressed  more  continuing 
motivation  than  those  who  worked  alone  for  activities  that 
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require  working  with  other  students,  £( 1,119)  =  5.82,  p.  <  .05, 
MSG  =  .94,  IS  =  .42. 

Discussion 

The  purpose  of  this  study  was  to  explore  the  effect  of 
cooperative  learning  and  type  of  reward  on  performance  and 
continuing  motivation.     Subjects  used  either  a  cooperative  or 
individual  learning  strategy  while  receiving  information, 
examples,  practice,  and  feedback  from  an  instructional  television 
lesson. 

Results  indicated  that  subjects  who  worked  alone  performed 
better  and  expressed  more  continuing  motivation  than  those  who 
worked  cooperatively.     These  findings  lend  support  to  other 
studies  which  suggest  that  a  cooperative  strategy  may  not  affect 
educational  outcomes  in  some  settings.     While  cooperative 
learning  has  influenced  student  performance  and  attitudes  in 
classroom  settings  (Johnson  &  Johnson,   1989;  Sharan,  1980; 
Slavin,  1990) ,  cooperative  learning  has  not  always  influenced 
performance  when  it  is  implemented  with  media  originally  designed 
for  individual  learning  (Carrier  &  Sales,   1987;  Klein  & 
Pridemore,  1992)  . 

The  results  of  the  current  study  may  have  occurred  because 
of  how  students  approach  learning  from  instructional  television 
lessons.     In  our  society,  television  viewing  is  an  individual 
experience  with  little  opportunity  for  interaction  among  viewers. 
Furthermore,  television  is  typically  implemented  for  individual 
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u;*>  in  instruction.     Most  students  have  very  little  practice 
working  with  other  people  when  using  instructional  television 
lessons . 

Even  though  subjects  in  the  triads  were  given  specific 
directions  for  implementing  a  cooperative  strategy  during  the 
lesson,   informal  observations  suggest  that  many  groups  did  not 
follow  these  directions.     After  each  segment  of  the  tape  was 
stopped,  subjects  in  several  of  the  triads  quietly  read  the 
question  in  the  workbook  to  themselves,     one  of  the  group  members 
would  usually  ask  the  others  for  the  answer  and  would  write  it  in 
the  workbook.     Very  little  on-task  discussion  occurred  in  these 
groups,     croups  that  finished  the  practice  before  other  triads  in 
the  room  would  usually  talk  about  topics  that  were  unrelated  to 
the  lesson.     These  behaviors  are  somewhat  different  than  those  of 
the  subjects  who  worked  alone.     After  reading  the  each  question 
and  answering  it,  many  individuals  who  finished  the  practice 
before  others  in  the  room  usually  reviewed  the  item  or  looked 
ahead  to  the  next  item  in  their  workbook.     This  additional  on- 
task  behavior  may  have  enhanced  the  performance  of  subjects  who 
worked  by  themselves. 

Another  explanation  for  the  results  found  in  this  study  may 
be  due  to  the  nature  of  the  reward  structure  provided.  Subjects 
in  both  the  cooperative  and  individual  learning  conditions  were 
placed  in  one  of  three  individual  reward  structures  (i.e.:  task, 
performance,  none).     Even  subjects  in  the  no  reward  conditions 
had  some  incentive  for  individual  learning,  because  they  were 
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?-«:Ld  that  the  lesson  would  increase  success  in  the  course.  None 
or  the  subjects  in  the  cooperative  groups  were  placed  in  a 
cooperative  reward  structure.     Slavin's  (1991),  suggestion  that 
group  rewards  have  a  strong  impact  on  cooperative  learning  in 
rlassroom  settings  may  also  be  legitimate  when  cooperative  groups 
u<se  media  that  were  originally  designed  for  individual  learning. 

This  study  has  some  implications  for  those  who  design 
instruction.     Educators  should  implement  specific  instructional 
elements  (e.g.,  practice,   feedback,  review)  when  they  present 
information  via  television  lessons.     Each  student  should  be  given 
the  opportunity  to  receive  these  elements  during  a  televised 
lesson.     Furthermore,  practitioners  who  wish  to  employ 
cooperative  learning  strategies  with  media  such  as  CAI  or 
television  should  provide  each  group  an  opportunity  to  practice 
working  with  others  before  assessing  their  performance.  Finally, 
educators  should  provide  appropriate  rewards  to  promote  group 
interaction  and  on-task  behavior  in  all  cooperative  learning 
settings. 

Future  research  should  continue  to  explore  the  use  of 
cooperative  learning  with  technologies  that  were  originally 
developed  for  individual  learning.     Studies  should  investigate 
how  different  reward  structures  (e.g.,   individual,  cooperative, 
competitive)   influence  outcomes  in  educational  technology 
settings.     Research  should  also  be  conducted  to  examine  how 
students  can  be  taught  to  cooperate  together  when  they  use  media. 
These  studies  should  examine  the  quantity  and  quality  of  group 
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interaction  when  cooperative  strategies  are  implemented  with 
media.     These  suggestions  will  assist  us  in  determining  the 
appropriate  use  of  cooperative  learning. 
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Table  1 

Mean  scores  and  standard  Bssiajagps  on  Escfaaaanss  Tssi 


Type  of  Reward 


Condition 


Individual  Learning- 


Cooperative  Learning 


Total 


Task 


11.5 
(3.2) 

9.9 
(3  0) 
10.7 
(3.2) 


Performance 


11.6 

(3.9) 

11.6 

(3.8) 

11.6 

(3.8) 


None 


12.5 

(4.2) 

10.0 

(3.9) 

11.3 

(4.2) 


Total 


11.9 

(3.7) 

10.  6 

(3.6) 

11.2 

(3.8) 


Note:  Maximum  possible  score  =  20; 
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Table  2 

mmti  Scores  and  Standard  Deviations  for  Continuing  Motivatipn 


Group 


Tf  OTTI* 
X  Lclll  K 

Individual 

Cooperative 

Total 

3 . 99 

3.73 

3.86 

(0. 86) 

(1.07) 

(0.98) 

2 .  61 

2.29 

2.45 

(1. 18) 

(1.11) 

(1.15) 

Question  3 

2.79 

2*43 

2.61 

(1. 18) 

M    1  7  \ 
1  X  •  X  /  J 

( 1 -  18 } 

Question  4 

2.61 

2.95 

2.78 

(1.21) 

,(1.40) 

(1.31) 

Question  5 

2.69 

1.86 

2.27 

(1.28) 

(1.00) 

(1.21) 

Question  6 

3.60 

4.02 

3.81 

(1.06) 

(0.87) 

(0.99) 

Question  7 

3.16 

2.57 

2.86 

(1.19) 

(1.19) 

(1.22) 

*Specific  item  description  are  found  in  text. 
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In  the  flurry  of  activity  surrounding  contractual  agreements  and 
installation  of  equipment  to  support  the  Channel  One  news 
broadcast,  many  people  have  neglected  to  ask  critical  questions 
concerning  the  implementation  of  this  innovative  educational 
program.     The  program  is  designed  to  be  used  as  a  complete, 
twelve-minute  broadcast,  but  the  literature  on  educational  media 
contains  ample  evidence  showing  that  teachers  frequently  use  only 
specified  parts  of  media  products  rather  than  entire  sequences  as 
they  are  packaged  (Cambre,  1987).     Past  experience  demonstrates 
that  rushing  to  place  technology  in  schools  without  adequate 
planning  can  lead  to  failure  of  an  innovation  (Cuban,  1986; 
Fullan  and  Pomfret,   1977;  Goodlad,   1975;  Knupfer,  1988;  Rogers, 
1983).     Like  any  other  educational  tool,  the  success  of  Channel 
One  will  not  rest  solely  within  the  product  itself  but  will 
depend  upon  how  it  is  received  by  the  key  people  who  use  it  and 
the  way  in  which  it  is  implemented. 

It  follows  then,  that  some  key  questions  need  to  be  addressed 
such  as:     Does  Channel  One  offer  a  product  that  is  worthwhile? 
Is  it  answering  a  need  in  the  curriculum?    How  do  the  teachers 
implement  the  12-minute  news  show?    What  are  teachers  doing  to 
reinforce  the  news  teaching  and  to  teach  critical  viewing  of 
advertisements?    In  order  to  determine  the  answers  to  these 
questions,   it  is  important  to  investigate  how  Channel  One  is 
being  utilized  within  schools  and  what  the  parents,  teachers,  and 
students  think  of  it. 

The  purpose  of  this  study  was  to  examine  how  students,  teachers, 
and  parents  reacted  to  the  Channel  One  news  broadcast  initially 
and  after  experience  with  the  program.  Students,  teachers,  and 
parents  in  three  school  districts  were  surveyed  to  gather 
information  about  how  the  news  broadcast  was  initiated  in  their 
school,  how  it  was  Implemented,  what  they  thought  about  Channel 
One,  and  their  habits  concerning  news  consumerism. 

These  groups  were  surveyed  twice;  once  at  the  beginning  of  their 
experience  with  Channel  One  and  once  at  the  end  of  the  school 
year  to  see  if  their  opinions  of  the  broadcast  or  their  news- 
related  activities  changed  after  experience  with  the  broadcast. 
Spot  checks  were  done  in  the  form  of  interviews  and  classroom 
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observations  to  see  how  the  broadcast  was  being  implemented  and 
if  it  had  any  effect  on  students'  interest  in  the  news. 

RESEARCH  QUESTIONS 

1.  Were  decisions  to  adopt  Channel  One  based  on  curriculum 
support? 

2.  Were  decisions  to  adopt  Channel  One  based  on  opportunism? 

3.  Who  made  the  decision  to  adopt  Channel  One? 

4.  What  factors  most  influenced  the  decision  to  adopt  Channel 
One? 

5.  How  has  the  broadcast  time  been  accommodated  within  the 
schedule  of  the  school  day? 

6.  How  are  teachers  implementing  the  content  of  Channel  One? 

7.  What  are  the  opinions  of  teachers,  students,  and  parents 
upon  initial  introduction  to  Channel  One  and  after 
experience  with  the  program? 

8.  Do  senior  high  school  students  react  any  differently  to 
Channel  One  than  do  junior  high  students? 


METHODOLOGY 


Subjects 

A  sample  of  high  schools  and  junior  high  schools  was  selected 
based  upon  which  schools  in  three  districts  received  the  Channel 
One  broadcast.    All  schools  that  received  the  broadcast  within 
the  three  chosen  districts  were  surveyed;  this  included  eight 
high  schools  and  seven  junior  high  schools. 

Cooperation  of  the  administration  was  secured  to  provide  for 
smooth  distribution  and  collection  of  survey  instruments.     It  wa 
hoped  that  this  strategy  would  provide  the  best  return  rate. 

Subjects  included  581  teachers,  2457  students,  and  652  parents 
from  junior  high  and  high  school  within  three  different  school 
districts.    The  subjects  were  selected  randomly  within  each 
school. 


Materials  and  Method 

Three  sets  of  written  survey  instruments  were  developed  for  use 
with  the  teachers,  students,  and  parents  respectively.  Each 
instrument  contained  questions  that  were  measured  on  a  five-point 
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Likert-type  scale  as  well  as  open-ended  questions  intended  to 
draw  out  more  information. 

Each  group  was  surveyed  twice,  once  after  two  weeks  of  experience 
with  the  Channel  One  broadcast  and  again  two  weeks  prior  to  the 
end  of  the  school  year.     Each  set  of  survey  instruments  for  the 
parents  and  students  remained  identical  between  the  pre  and  post 
experience.     The  post  survey  instrument  used  for  the  teachers  was 
slightly  different  from  that  used  in  the  pre  survey. 

Surveys  were  distributed  to  school  principals  who,  in  turn, 
requested  the  cooperation  of  the  teachers  in  gathering  the  data. 
Students  surveys  were  distributed  to  entire  classes  of  students 
and  collected  immediately  upon  completion.     Students  who  were 
surveyed  were  asked  to  take  a  survey  instrument  home  to  their 
parents.     Return  rates  were  one  hundred  percent  for  students, 
about  95%  for  teachers,  and  about  25%  for  parents. 

Analysis 


Data  analysis  was  conducted  with  summary  and  nonparametric 
statistics.     These  allowed  accurate  comparisons  to  be  made  and 
visually  diagrammed  for  presentation.     Information  collected  by 
interview  was  analyzed  qualitatively. 

RESULTS  «  CONCLUSIONS 

The  decision  to  adopt  Channel  One  was  based  on  opportunism  rather 
than  on  curricular  content  or  a  specific  curricular  need. 
Schools  were  interested  in  receiving  something  at  a  low  cost,  and 
personnel  were  especially  excited  about  the  possibilitv  of 
receiving  equipment  and  cabling.     In  some  cases  teachers  were 
disappointed  because  the  equipment  configuration  prevented  them 
from  doing  the  type  of  activity  that  they  had  envisioned. 

X"«.?af?  district,  the  superintendent  or  the  superintendent  along 
with  the  school  board  made  the  decision  to  allow  channel  One 
within  the  district.     Each  superintendent  then  allowed  the  final 
decision  to  be  made  by  the  individual  school  principals.     Some  of 
the  principals  consulted  with  the  teachers  and  others  did  not. 

The  teachers  had  little  investment  and  did  not  seem  concerned 
about  making  a  serious  effort  to  implement  the  broadcast  in  a 
meaningful  way.     Those  teachers  who  were  involved  in  1-he  decision 
to  adopt  Channel  one  seemed  to  be  making  a  better  effort  to  make 
it  successful  than  those  who  were  not  consulted  prior  to 
implementation.     In  some  schools  the  staff  believed  that  the  real 
value  of  Channel  one  was  not  in  learning  about  the  news,  but 
instead  was  related  to  the  outgrowth  of  activities  attributed  to 
the  Channel  one  broadcast.     For  example,  some  schools  used  the 
system  for  school  announcements,  one  school  started  its  own 
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within-school  news  program,  and  another  one  extended  that  idea 
outside  of  the  school  to  support  community  events. 

The  twelve  minutes  of  time  was  handled  in  various  ways  but  all 
schools  tried  to  implement  the  program  in  a  way  that  did  not 
subtract  time  from  existing  subjects.    One  district  added  twelve 
minutes  to  the  school  day  for  each  school  using  Channel  One.  The 
other  two  districts  provided  the  twelve  minutes  by  a  combination 
of  shortening  passing  time  between  classes,  shortening  lunch 
period,  or  using  homeroom  time  for  the  Channel  One  broadcast. 


in  general,  students  at  the  junior  high  level  liked  the  broadcast 
more  than  did  the  high  school  students.     Teachers  had  a  more 
favorable  assessment  of  the  broadcast  than  did  the  students. 
Parents  did  not  seem  to  know  much  about  the  broadcast.  More 
parents  responded  to  the  second  round  of  the  survey  which 
suggests  that  more  parents  became  more  aware  of  Channel  One  or 
possibility  more  attentive  to  Channel  One  issues  during  the 
school  year. 

In  most  cases,  Channel  One  was  implemented  as  a  stand-alone  topic 
rather  than  in  accordance  with  curricular  goals.    There  appeared 
to  be  little  class  time  devoted  to  preparing  the  students  for  the 
broadcast  or  for  discussions  following  the  broadcast.  Students' 
attention  to  and  involvement  with  reading  newspapers  did  not  seem 
to  increase  over  the  course  of  the  school  year,  but  students 
appear  to  watch  television  news  more  frequently. 

The  attached  graphics  represent  the  percentage  of  respondents 
that  agreed  or  strongly  agreed  with  each  listed  topic.     It  is 
evident  from  the  graphics  that  initially  about  70%  of  teachers 
and  50%  of  students  agreed  that  Channel  One  was  easier  to 
understand  than  other  television  news  broadcasts,  and  each  of 
those  percentages  increased  by  approximately  5%  after  experience 
with  the  broadcast.      After  experiencing  the  broadcast,  a  higher 
percentage  of  students  and  parents  believed  that  advertisements 
can  have  too  much  influence  on  students;  teachers  were  the 
opposite. 

In  general,  as  teachers  and  students  gained  experience  with 
Channel  One  they  were  less  inclined  to  believe  that  it  teaches 
critical  consumerism.    When  asked  if  the  product  is  a  very  good 
qualify  teaching  tool,  teachers  were  the  more  enthusiastic  than 
parents  and  students,  and  more  of  teachers  agreed  with  this 
statement  after  experience  with  the  product;  students  and  parents 
were  the  opposite.     In  response  to  a  query  about  whether  their 
school  should  continue  Channel  One,  teachers  and  high  school 
students  seemed  to  lose  some  interest  while  junior  high  students 
gained  enthusiasm  for  the  broadcast  during  the  schoolyear;  a 
clear  majority  of  all  groups  agreed  that  their  school  should 
continue. 
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In  the  interest  of  saving  space  in  this  publication,  I  will  not 
discuss  each  of  the  graphics,  but  I  shall  attach  selected 
graphics  along  with  samples  of  the  written  survey  instruments  for 
the  reader  to  study. 
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DEMOGRAPHICS 

Round  One 

Teachers  ,  students  Parents 

Junior  High       275  of  280  1105  58 

High  8chOOl         3  0«  Of  32  0  1322  82 

TOTAL  581  2427  140 

Teachers  students  Parent* 

Junior  High       272  of  280  1118  316 

Senior  High       301  of  320  1339  336 

TOTAL  573  2457  652 
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Easier  to  Understand  thin  Other  TV  News  Teaches  Current  Events  Very  Well 


Teachers 
Students 
Parents 


ca.  2  wks.  ca.  9  mos.  ca.  2  wks.  ca.  9  mot. 

Degree  of  Experteoco  Degreo  of  Experience 


Teaches  Geography  Very  Well 


ca.  2  wfcs.  ca.  9  mos. 

Degree  of  Experience 


-a-  Teachers 

Students 
Pafams 


Students  Learn  a  Lot 


ca.  2  wks.  ca.  9  mos. 

Degree  of  Experience 


Taacnars 

Junior  High 
■•>  Senior  High 


Students  Seem  Bored 


CM.  2  wfcs.  ca.  •  tr 

Degree  of  Cupar  fence 


Teachers 

Junior  High 
Senior  High 


Channel  One  is  Too  Simple 


ca.  2  wks.  ca.  9  mos. 

Degree  of  Experience 


Teachers 

Junior  High 
Senior  High 
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Too  Complex  for  Short  Time 


-**  TMCtW» 

JuntorHlQh 
-*  SentofHtgh 


ol  2  wks.  CM.  9  mos. 

Pegcas)  of  Experience 


Encoursges  Thinking  About  the  News 


■*  Teecfte* 

Students 
•*  Parents 


ca.  2  wks.  ca.  9  mot. 

Degree  of  Experience 


Discuss  Ads 


ca.2wks.  ca.dmos. 
D«gree  of  Experience 


-*  Teachars 
Students 
Parents 


Nor 


Conwnc**  ft  r#*c/*s  Crtf/ca/  ConaumeYiam 


•ev  Teachers 
Students 
-»  Parents 


ca-2wks.  ca.9mos. 
Otgrt*  of  Experience 


Ads  Have  Too  Much  influence 


CM.  2  W*S. 


ca.  9  mos. 


Degree  of  Experience 


Taachers 
-e-  Students 
-e>  Parents 


Ads  Seme  es  Those  on  TV 


cm.  2  wks.  ca.  9  mos. 

OMQtm  of  Experience 


-e>  Teachers 

-e-  Students 
-e>  Parents 
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Ads  are  Pretty  Truthful 


*♦  Taachors 
Students 
*•*  Paranta 


ca.  2  wkt.  ca.  9  mot. 

Dagroa  of  Exparianca 


4tfs  Don  1  Belong  in  School 


*•>  Taachars 
Siudantt 
Par»m* 


ca.  2  wk$.  ca.  9  mo$. 

Dagraa  of  Exparfaoca 


Causes  Students  to  Reed  Mors  Now*  Uagaainoa 


Taachars 
"•"  Studants 
Pa/ tots 


ca.  2  wkt.  ca.  9  mot. 

Oagraaof  Expartanca 


Re sd  News  Magazines  at  Least  3  Times  Weekly 

45- 


Taacntft 

Studantt 

Pa#tnis 


ca.  2wks.  ca.  9mos. 

Dagraa  of  Expaeianca 
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Raad  Nawapapar  At  Laast  3  Tlmaa  Waakly 

90  - 


Watch  Naws  at  Laast  3  Timas  Par  Waek 


ca.  2  wks,  ca.  9  mot. 

Dagraa  of  Expartanca 


Taachara 
Studanta 
Params 


ca.  2  wks.  ca.  9  rnos. 

Dagraa  of  Exparianea 


h*.  Taacftars 
-a-  Studanta 
Paranta 


Vary  Good  Quality  Taachlng  Tool 


I  Lika  Channal  Ona  a  Lot 


90- 

B0- 

70- 

I 

* 

60- 

a 

50- 

40- 
30- 

ca.  2  wks.  ca.  9  mos. 

Dagraaof  Exparianca 


-•■  Taacnars 
Studanta 
■*  Parants 


ca.  2  wks.  ca.  9  mos. 

Oaoraaof  Exparianca 


Taacnars 
Junky  Hfch 
SankxHiQh 


c 

i 


School  Should  Continua  Channal  Ona 

90  - 


Should  Continua  Channal  Ona 


Taacnars 

-a-  Studsnts 
-a>  p  if  ants 


ca.  2  wks.  ca.  9  mos. 

Dagraa  of  Exparianca 


i 
i 


ca.  2  wks.  ca.  9  mos. 

0*qtm  of  Exparlanc* 


TaacMA 

Junhx  High 
Sankx  Htgfi 


a 
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The  Advertising: 


Strongly  ***      M  Strong 

Disagree   Disagree   Sure     Agree  Agree 


1 .  Does  not  belong  in  schools. 

2.  Can  have  too  much  Wluence  on  students. 

3.  la  pratty  truthful  about  the  product. 

4.  Pratsuras  studanta  to  buy  product*  lika  othara  have. 

5.  Our  classroom  discusees  tha  edvertieementt. 

6.  Channel  One  ada  are  Uke  the  ada  ahown  on  regular  TV. 

7.  Studanta  seem  to  like  the  ada  better  than  the  newa. 

8.  Student*  remember  the  adverting  more  than  the  newa. 

9.  There  it  nothing  wrong  with  watching  ada  in  echooL 

10.  I  sea  ads  In  other  placee  in  achool.  not  Just  Channel  One. 

Write  a  brief  explanation  for  each  following  question. 

1 .  What  problems  could  arise  out  of  showing  advertisements  in  schools? 


3 
3 
3 

3 
3 
3 
3 
3 
3 
3 


5 

5 

5 

5 

5 

5 

5 
5 
5 
5 


2.  Do  you  think  it  Is  okay  to  show  advertisement*  in  school?  Why  or  why  not? 

3.  CHBCJUII  who  were  involved  in  making  the  decision  to  Implement  Chennel  One? 

_  Superintendent   School  Board  _  Principal  „ Teacher.   Parents   Student. 

4.  What  factors  influenced  that  decision? 

5.  Why  do  you  think  Channel  One  has  caused  such  a  controversy?  Please  explain. 

6.  in  your  opinion,  what  Is  the  key  Issue  that  educetcrs  should  be  concerned  ebout? 

7.  Are  you  attempting  to  fit  the  program  Into  the  curriculum  or  Is  It  being  used  as  a  stand-alone  topic?  (eh**  t) 

 Fit  into  curriculum.  HOW?  . — ,  —  

 Stand-alone  topic  m 

8.  What  usually  happens  in  your  classroom  after  the  Chennel  One  broadceet  Is  over? 


9.   How  has  your  school  accommodated  the  extra  12  rrtnu.es  to  view  Channel  On.  news?  H  there  has  been  any  change 
in  daily  schedule,  how  do  you  feel  about  it? 
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i  ne 

Strongly 

Disagree  Disagree 

Not 
Sure 

Soongty 
Agree  Agree 

1. 

Dots  not  belong  in  school*. 

1  2 

3 

4  5 

2. 

Can  have  too  much  influence  on  students. 

1  2 

3 

4  5 

3. 

It  pretty  truthful  about  the  product. 

1  2 

3 

4  5 

4 

Pressures  students  to  buy  product*  Ifce  othan  hava. 

1  2 

3 

4  5 

5. 

Our  classroom  diacuataa  tha  edvertieememe. 

1  2 

3 

4  5 

6. 

Chsnnel  Ona  ada  sra  Uta  tha  ads  shown  on  raguiar  TV. 

1  2 

3 

4  5 

7. 

Students  seem  to  Uke  tha  ads  better  than  the  news. 

1  2 

3 

4  5 

0. 

Students  remember  the  advertising  more  than  the  news. 

1  2 

3 

4  5 

9. 

There  Is  nothing  wrong  with  welching  eds  in  school 

1  2 

3 

4  5 

10. 

I  see  ads  in  other  pieces  in  school,  not  just  Channel  One. 

1  2 

3 

4  5 

Writ©  •  brief  explanation  for  tach  following  question. 

1. 

Describe  your  school's  experience  with  Channel  One  this  yetr. 

In  general? 


Problems? 

Successes? 

Inspirations? 

2.  Has  the  srn>wingo<  advertisements  In  sen  I  N0  I  1  YES 

Why  or  why  not? 

3  Are  you  attempting  to  fit  the  program  Into  the  curriculum  or  Is  it  being  used  as  a  eland-alone  toplc?-<chec* 

 Fit  into  curriculum.  HOW?_  

 Stand-atone  topic 

4.  What  usually  happens  in  your  classroom  after  the  Channel  One  broadcast  Is  over? 

5.  Are  any  particular  teachers  responsible  tor  discuseing  the  news  with  students  after  viewing?  Plaate  explain. 
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What  is  your  gender?  (check  on«)  Male   Female 

Write  a  brief  explanation  tor  each  question  below. 
1 .  What  problems  could  arise  out  of  showing  advertisements  in  schools? 


2.  Do  you  think  it  is  okay  to  show  advertisements  in  school?  Why  or  why  not? 

3. .  From  yesterday's  Channel  One  news  show ,  name  3  news  stories  and  tell  why  they  are  important. 


(2) 


(3) 


4.  From  yesterday's  Channel  One  news  show  .  name  and  describe  3  advertisements  that  you  remember  the  most. 
(D 


(2) 


(3) 


5.  List  the  news  magazines  that  you  read  regularly. 


6.  Whicli  pans  of  the  newspaper  do  vou  read? 


7.   What  usually  happens  in  your  classroom  alter  the  Channel  One  broadcast  is  over? 
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School 


Your  Occupation . 


Channel  Ona  Naws  Survty  for  Parents 


Name  (optional). 
Gtndtr  


Clrcia  a  numbar  for  tach  statamant  below. 

1.  Have  you  hurt  of  Channel  One  ntws  before? 

2.  Hm  you  taan  any  of  tha  Channel  Ona  programs? 

Channel  Ona  News: 

1.  is  easier  for  students  to  understand  then  other  TV  news. 

2.  HeJpe  students  become  culturally  literete. 

3.  Helpo  students  become  better  citizens. 
*•  Teaches  students  about  our  government 

5.  Teaches  students  about  health. 

6.  Teaches  studentr-  to  be  chticaJ  consumers. 

7.  HeJpe  students  become  critical  thinkers. 

a.  Leads  students  to  read  more  news  magazines. 

9.  Leads  students  to  read  more  newspapers, 

10.  Answers  a  need  in  the  curriculum. 

1 1 .  Teaches  current  events  very  v  -  . 

12.  Teaches  geography  very  well 

13.  is  a  very  good  quality  teaching  tool 

14.  Students  team  a  bt  from  Channel  One  newt. 

1$.  ChanneiOne  news  «  a  good  addition  to  the  school  curriculum. 

16.  I  want  my  chU  to  view  Channel  One  news. 

17.  Students  who  view  Channel  One  news  w*  have 

an  educational  advantage  over  those  who  don't  view. 

18.  Channel  One  dictates  school  curriculum  and  should  not  be  allowed 

The  Advertising: 

1.  Does  not  belong  in  schools. 

Z  Can  have  too  .nuch  influence  on  students. 

X  Is  usually  pretty  truthful  about  the  products. 

4.  Pressures  students  to  buy  products  lite  others  have. 

5.  Channel  One  ads  are  tike  the  ads  shown  on  regular  TV. 

6.  There  is  nothing  wrong  with  wesehing  ads  in  school 
7. 
I. 
9. 

10. 


Yes 


Strongly 
Disagree 


11. 


I  see  ace  in  other  pfecee  in  school,  not  just  Channel  One. 

Teenagers  are  guHMe  and  should  not  be  view  ada  in  school 

Advertising  to  teenagers  in  school  should  not  be  alio*  «d. 

The  benefits  of  Channel  One  news  outweigh  the 
disadvantages  of  advertising. 

Teenagers  are  old  enough  to  be  critical  consumers  of  sda. 


No 

2 

2 

Disagree 

2 

2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


Strongly 

Disagree  Disagree 

2 


Date. 
Age_ 


Not 
Sure 


Agree 


Not 
Sure 


Agree 


Strongly 
Agree 
S 

5 

S 

5 

5 

5 

5 

5 

S 

5 

5 

5 

5 

5 

5 

5 

5 


Strongly 
Agr<te 
5 

5 

5 

5 

5 

5 

5 

5  * 

5 

5 


TURN  THIS  PAGE  OVER  AND  FILL  IN  THE  BACK 
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Ntver 


In  any  ggftk,  how  ofttn  do  you: 

1.  Watch  the  new*  on  TV  at  home? 

2.  Reed  the  newapeper  ? 

3.  Read  news  magazine*? 

4.  Discuss  the  newt  with  your  ehW(ren)? 

5.  Criticafty  discuss  advertisements  with  your  chiid(ren)? 

Write  a  brlaf  explanation  for  each  following  quastion 
1 .  What  problems  could  arise  out  of  showing  advertisements  in  schools? 


1-2 
Times 
2 

2 

2 

2 

2 


Times 
3 

3 

3 

3 

3 


54  Every 
Trnes  Day 
5 

5 

5 

5 

5 


Z  Do  you  think  it  la  okay  to  show  advertisements  in  school?  Why  or  why  not? 

3.  Were  mi  involved  In  making  the  decision  to  implement  Channel  One?   Yes   No 

4.  What  factor*  Influenced  that  decision? 

5.  Why  do  you  think  Channel  One  has  caused  such  a  controversy?  Please  explain. 


6.    In  your  opinion,  what  is  the  kay  Issue  that  tducators  should  be  concerned  about  when  considering  Channel  One' 


7.    How  has  your  school  accommodated  the  extra  12  minutes  to  view  Channel  One  news?  If  there  has  been  any  change 
In  daily  schedule,  how  do  you  (eel  about  it? 


8.  Highest  grade  you  completed  in  school? 

 Did  not  complete  High  School   Completed  High  School   Some  1-2  years  of  couege.no  degree 

 2  yrCoflege  or  Vocational  School  Degree   4  yr  collage  degree  _J3raduate  School 

9.  What  is  your  combined  household  annual  Income  range?   Less  than  15.000   15.000*25.000 

 25,000*35.000   35.000-45.000   45,000-55.000   55.000-55.000   More  than  65.000 

10.  Check  your  main  ethnic  origin.  Asian  Black  Hispanic  Native  American  White 

THANK  YOU  FOR  YOUR  HELP! 

PLEASE  RETURN  THIS  SURVEY  TO  SCHOOL  WITH  YOUR  CHILD. 
IF  YOU  DESIRE,  FEEL  FREE  TO  PLACE  IT  IN  A  SEALED  ENVELOPE, 
LABELED  "CHANNEL  ONE  SURVEY" 
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Introduction 

Steve  Jobs  has  suggested  that  the  1980s  were  the  age  of  personal  computing  while 
the  1990s  will  be  the  age  of  interpersonal  computing.  The  advent  of  interpersonal 
computing  has  enormous  implications  for  work  and  learning.  Schrage  (1991)  reports  that 
"new  technologies  of  collaboration  are  emerging  that  will  radically  transform  the  way 
people  share  their  thoughts."  Computer  networks  have  provided  a  foundation  for 
interpersonal  computing,  but  as  Schrage  mentions,  new  tools  are  emerging;  the  centerpiece 
is  what  is  called  "Groupware."  This  paper  will  examine  this  technology  as  well  as  the 
theoretical  framework  that  will  underlie  interpersonal,  collaborative  computing.  First,  an 
examination  of  Groupware. 
Groupware 

What  is  "Groupware"?  Fraase  (1991)  defines  groupware  "as  a  group  of  technologies, 
techniques,  and  services  designed  to  help  people  collaborate  more  effectively,  productively, 
and  creatively.  Groupware  can  consist  of  hardware,  software,  services,  and  support." 
Groupware  is  based  upon  the  convergence  of  networking  and  multimedia  technologies. 
This  collaborative  computing  utilizes  networking,  communications,  concurrent  processing, 
and  windowing  environments.  Hypermedia  can  also  be  integrated.  Specific  media  such  as 
voice  and  video  can  play  major  roles  in  collaborative  computing,  although  the  limited  ability 
of  computers  to  recognize  or  manipulate  their  content  complicates  their  full  integration. 

Beyond  technical  definitions,  groupware  is  about  groups  of  people  managing  their 
collective  information  resources  (Fraase,  1991).  The  collaborative  context  appears 
wherever  people  are  working  together  to  add  value.  Groupware  is  intended  to  create  a 
shared  workspace  that  supports  dynamic  collaboration  in  a  work  group  over  space  and  time 
constraints.  Supporting  team  collaboration  over  time  is  the  central  purpose  of  most 
groupware.  However,  groupware  can  support  many  activities.  Johansen  (1988)  identifies 
seventeen  functions  that  groupware  can  support: 

•  Face-to-face  meeting  facilitation 

•  Group  decision  support 

•  Computer-based  telephony  extensions 

•  Presentation  support 
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•  Project  management 

•  Calendar  management 

•  Group-authoring 

•  Computer-supported  face-to-face  meetings 

•  Screen-sharing 

•  Computer  conferencing 
Text  filtering 

•  Computer-supported  audio/video  teleconferencing 

•  Conversational  structuring 

•  Group  memory  management 

•  Spontaneous  interaction 
Comprehensive  workgroup  support 

•  Nonhuman  meeting  participants  (using  intelligent  agents) 

Different  groupware  systems  emphasize  different  capabilities.  Systems  that  are 
commercially  available  tend  to  emphasize  computer  networking  capabilities.  Some 
groupware  packages  enhance  e-mail  with  features  that  help  route  "compound '  documents 
among  coworkers  collaborating  on  a  document.  At  a  higher  level,  groupware  is  much 
more  than  enhanced  e-mail  —  it's  a  multiuser  database  that  can  be  used  for  group  projects. 
More  advanced  collaborative  groupware  packages  permit  the  creation  of  compound 
documents  made  up  of  several  types  of  media.  Group-created  documents  that  are  often 
limited  to  text  and  graphics  will  increasingly  take  on  other  data  types,  such  as  sound,  still 
video,  animation,  and  movies  (Fraase,  1991).  Experimental  groupware  systems  under 
study  in  research  labs  in  Japan,  europe,  and  the  United  States  exemplify  these  more 
sophisticated  capabilities.  For  example,  the  Xerox  PARC  Media  Space  emphasizes 
videoconferencing,  including  a  computer-based  VideoVVhiteboard  that  serves  as  a  drawing 
tool.  This  VideoVVhiteboard  is  described  as  "a  high  tech  version  of  shadow  puppet  theater 
(Brittain,  1991)."  It  is  an  electronic  screen  that  can  be  used  for  writing  notes  and  drawing 
diagrams,  just  like  a  traditional  whiteboard  (or  blackboard).  This  VideoVVhiteboard  can  be 
seen  —  and  written  upon  —  by  coworkers  at  different  sites  simultaneously.  For  a  detailed 
description  of  groupware  systems,  refer  to  McLellan  &  Knupfer  (In  press). 
Collaboration 

Groupware  technology  provides  a  platform  to  support  collaboration  and  other  forms 
of  interpersonal  computing.  In  order  to  support  collaboration  via  technology,  it  is  essential 
to  understand  what  constitutes  collaboration.  Collaboration  is  not  merely  communication. 
Schrage  (1991)  defines  collaboration  as  "the  process  of  shared  creation:  two  or  more 

587 

ERIC  OJo 


0 

:RIC 


Groupware 
4 

individuals  with  complementary  skills  interacting  to  create  a  shared  understanding  that  none 
had  previously  possessed  or  could  have  come  to  on  their  own.  Collaboration  creates  a 
shared  meaning  about  a  process,  a  product,  or  an  event.  In  this  sense,  there  is  nothing 
routine  about  it.  Something  is  there  that  wasn't  there  before.  Collaboration  can  occur  by 
mail,  over  the  phone  lines,  and  in  person.  But  the  true  medium  of  collaboration  is  other 
people.  Real  innovation  comes  from  this  social  matrix." 

According  to  Schrage,  "Quality  collaboration  —  the  kind  of  efforts  C;*t  have  driven 
breakthroughs  in  science,  the  arts,  and  technology  —  occurs  with  neither  the  frequency  nor 
the  intensity  it  should,  in  part  because  there  are  few  tools  explicitly  designed  to  encourage 
or  support  it."  Groupware  offers  the  potential  to  support  collaboration,  but  in  designing 
and  implementing  groupware,  it's  essential  to  understand  both  the  technology  and  the 
nature  of  collaboration.  Norman  (1991)  points  out  that  the  same  principles  that  apply  to  the 
development  of  computer  systems  for  individuals  are  not  sufficient  for  computer  systems 
rbr  groups.  It  is  not  these  design  principles  do  not  apply.  Rather,  group  activity  is  vastly 
different  from  individual  activity  and  has  its  own  needs  and  requirements.  "Technology 
alone  cannot  provide  the  answers  when  we  deal  with  human  activities.  The  tasks,  the 
culture,  the  social  structure,  and  the  individual  human  are  all  essential  components  of  the 
job,  and  unless  the  computational  tools  fit  "seamlessly"  within  this  structure,  the  results 
will  be  failure  (Norman,  1991)." 

So  just  what  is  collaboration?  Los  Angeles  Times  reporter  Michael  Schrage  (1991) 
has  written  an  insightful  book,  Shared  minds:  The  new  t^hnni^j^  nf  r.n\\*hnru,inn  that 
explores  this  issue  in  depth:  "Collaboration  describes  a  process  of  value  creation  that  our 
traditional  structures  of  communication  and  teamwork  can't  achieve."  The  driving  spirit 
behind  any  collaboration  is  always  dynamic;  it  should  never  stagnate.  Schrage  identifies 
two  types  of  collaboration:  formal  and  informal  collaboration.  Formal  collaboration  is  the 
process  we  follow  in  highly-structured  settings  such  as  meetings  and  task  forces.  This 
type  of  collaboration  revolves  around  structures,  procedures,  and  formal  processes. 
Informal  collaboration  is  characterized  by  less-formal  settings  and  less  structured,  more 
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casual  interactions,  for  example,  crossing  paths  in  hallways,  at  the  coffee  machine,  over 
lunch.  Informal  collaborations  are  much  less  structured  than  a  formal  collaboration.  Good 
collaborators,  regardless  of  the  type  of  collaboration  employed,  "accept  and  respect  the  fact 
that  other  perspectives  can  add  value  to  their  own  (Schrage,  1991)."  Schrage  suggests  that 
we  collaborate  not  only  with  our  co-collaborators  but  with  the  symbols  people  create. 

Schrage  identifies  collaboration  goals  or  outcomes  and  collaboration  constraints. 
Collaboration  may  result  in  (1)  Problem  solving;  (2)  Creativity;  and/or  (3)  Discovery. 
Collaboration  constraints  include:  (1)  expertise;  (2)  time;  (3)  money;  (4)  competition;  and 
(S)  conventional  wisdom  and  common  sense. 

> 

According  to  Schage,  there  are  thirteen  design  themes  that  characters  collaboration, 
including:  (1)  Competence;  (2)  A  shared,  understood  goal;  (3)  Mutual  respect,  tolerance, 
and  trust;  (4)  Creation  and  manipulation  of  shared  spaces;  (5)  Multiple  forms  of 
representation;  (6)  Playing  with  the  representations;  (7)  Continuous  but  not  continual 
communication;  (8)  Formal  and  informal  environments;  (9)  Clear  lines  of  responsibility  but 
no  restrictive  boundaries;  (10)  Decisions  do  not  have  to  be  made  by  consensus;  (1 1) 
Physical  presence  is  not  necessary;  (12)  Selective  use  of  outsiders  for  complementary 
insights  and  information;  and  (13)  Collaboration's  end 

Groupware  offers  the  potential  to  support  all  of  these  design  themes.  Design  theme 
Four  —  Creation  and  manipulation  of  shared  spaces  —  is  especially  relevant  to  a 
groupware  collaboration  context.  According  to  Schrage:  "All  collaborations  rely  on  a 
shared  space.  It  may  be  a  blackboard,  a  napkin,  a  piano  keyboard,  a  rehearsal  room,  a 
prototype,  or  a  model.  Independent  of  whether  the  collaborators  are  artists,  scientists, 
professionals,  managers,  or  mechanics,  collaborators  are  inevitably  drawn  to  a  shared 
space  to  share  the  ideas  and  insights  that  will  solve  the  problem  or  achieve  the  task.  The 
shared  space  becomes  a  partner  in  collaboration."  Groupware  readily  offers  multiple  forms 
of  representation  and  this  capability  will  increase  as  the  technology  advances.  Related  to 
this,  groupware  offers  collaborators  the  capability  of  playing  with  the  representations. 
Continuous  but  not  continual  communication  is  made  highly  convenient  in  a  groupware 
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environment.  Some  other  factors  that  underlie  collaboration  include  the  following:  (1) 
Conflicts  are  not  two-sided;  (2)  All  problems  are  interdisciplinary;  (3)  Solutions  can  be 
reached  by  widening  the  community;  (4)  Consensus  doesn't  have  to  mean  unanimous 
agreement;  (5)  Voting  doesn't  work;  consensus  building  does;  and  (6)  Sometimes  it's  best 
to  just  agree  rather  than  trying  to  agree  on  why  you're  agreeing  (Schrage,  1991). 

The  collaborative  process  can  be  broken  down  into  two  distinct  elements:  conceptual 
collaboration,  and  technical  collaboration  (Schrage,  1991).  Conceptual  collaboration  is  the 
process  of  defining  the  overall  goal  of  the  collaboration.  This  must  precede  the  technical 
collaboration.  An  example  of  conceptual  collaboration,  according  to  Schrage,  is  when  a 
film  director  and  a  cinematographer  block  the  action  and  camera  angles  before  filming 
begins.  Technical  collaboration  is  the  actual  attempt  to  solve  the  problem  that  was  defined 
in  the  conceptual  collaboration.  This  is  apt  to  be  a  process  of  repeated  attempts.  A  good 
example  of  technical  collaboration  is  James  Watson  and  Francis  Crick  working  on  the  DNA 
model  to  establish  the  structure  of  DNA  (the  double  helix),  based  on  their  conceptual 
model. 

Interpersonal  computing  is  qualitatively  different  from  personal  computing.  Norman 
(1991)  emphasizes  that  this  qualitative  difference  should  inform  the  design  of  interpersonal 
computing  tools.  Other  analysts  speculate  about  the  social  impacts  of  interpersonal 
computing.  For  example,  Perin  (191)  explains  that  etetronic  social  fields  -  interpersonal 
computing  —  may  threaten  managers'  assumptions  about  conventional  bureaucratic 
organizations  because  they  call  into  question  organizational  authority,  they  cross  functional 
divisions,  and  they  create  options  for  the  times  and  places  of  work.  So  that  while  the 
virtual  groups  linked  by  groupware  may  enhance  innovation  and  productivity  in 
bureaucracies,  interpersonal  computing  may  be  susceptible  to  managerial  suspicion  and 
negativism.  This  is  an  issue  that  will  have  to  be  addressed  in  the  implementation  of 
groupware  systems  within  organizations. 

According  to  Schrage,  it  is  essential  for  individuals  and  institutions  to  make  the 
adjustment  to  interpersonal,  collaborative  computing:  "As  individuals,  we  each  possess 
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special  skills  and  expertise,  plus  the  desire  to  apply  them  productively.  Yet  to  a 
disturbingly  large  degree,  most  organizations  lack  the  collaborative  infrastructures  that 
enable  people  to  share  their  talents  in  ways  that  satisfy  the  individual's  need  for  expression 
and  the  organization's  imperative  for  results.  As  a  result,  people  feel  increasingly 
frustrated,  and  the  organizations  that  employ  them  moan  about  declining  productivity  and 
shriveled  morale.  There's  no  balance."  Groupware  tools  offer  the  potential  to  redress  the 
balance  so  that  collaboration  can  flourish. 
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Restructuring  Library  Media  Centers  in  Kentucky 

As  a  result  of  the  Kentucky  Education  Reform  Act 
(KERA)  of  1990,  schools  in  Kentucky  are  restructuring 
through  site  based  management,  non-graded  primary 
schools,  performance-based  student  assessment,  and 
instructional  uses  of  technology  in  order  to  change  the 
way  children  are  educated  within  the  Commonwealth, 
Within  KERA,  technology  is  viewed  as  playing  an 
important  role  in  enlarging  and  enriching  the  learning 
experiences  of  students.    Thus,  the  need  has  arisen  to 
develop  a  long  range  plan  for  the  efficient  and 
equitable  use  of  technology  at  all  levels.     In  order  to 
facilitate  the  integration  of  technology  in  education, 
KERA  created  a  state  advisory  council  for  educational 
technology  whose  purpose  it  is  to  design  and  implement 
a  plan  to  cover  all  aspects  of  educational  technology. 

The  initiation  of  the  Kentucky  mandate  has  forced 
each  school  to  quickly  prepare  a  five-year  technology 
plan.  For  many,  technology  is  a  new  area;  one  that 
because  of  a  lack  of  funds  for  many  years,  has  been 
largely  neglected  in  terms  of  purchases  as  well  as 
serious  utilization  studies  by  individual  schools  and 
their  library  media  specialists.    Therefore-  such  a 
sweeping  reform  as  KERA  presents  an  opportunity  to 
examine  not  only  the  implementation  of  new 
technological  devices  but  also  to  track  any  potential 
changes  in  the  role  of  the  library  media  specialists. 
Preliminary  to  a  long-range  examination,  a  description 
of  the  schools'  current  technological  sophistication 
and  the  roles  of  school  personnel  becomes  important. 
Specifically,  determinations  are  required  of  the 
technologies  and  technology  expertise  currently  within 
the  schools,  and  the  role  of  the  library  media 
specialists  in  the  initial  planning  and  implementation 
stages. 

This  paper  presents  the  results  obtained  in  a  1992 
survey  of  library  media  specialists  in  southeastern 
Kentucky.     In  particular,  the  paper  focus  on:  a) 
technological  resources  currently  available  in  library 
media  centers;  and  b)  the  library's  media  specialists' 
perception  of  their  role  and  in  the  future  as  a  result 
of  the  changes  brought  about  by  the  Kentucky 
Educational  Reform  Act. 

Survey  Results 
The  results  of  the  survey  indicated  that  the  four 
most  widely  available,  newer  technologies  are:  a) 
televisions  (98%),  b)  video  cassette  recorders  (97%), 
c)  computers  (71%),  and  d)  camcorders  (58%).  In 
addition,  collections  of  videotapes  and  software 
programs  are  being  housed  in  library  media  centers. 
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The  equipment  and  accompanying  instructional  materials 
are  most  often  checked  out  of  the  library  and  utilized 
by  the  teachers  in  the  classroom  to  present  information 
to  students.    Four  technologies  existing  in  less  than  a 
third  of  the  library  media  centers  are:  a)  CD-ROM's 
(24%),  b)  automated  library  systems  (16%),  c)  facsimile 
machines  (11%),  and  d)  videodisc  players  (2%). 

As  a  result  of  KERA,  schools  are  beginning  to 
develop  their  five  year  technology  plans.     The  five 
most  frequently  mentioned  technologies  desired  are:  a) 
computers  (44%),  b)  CD-ROM's  (36%),  c)  networking  of 
computers  (35%),  d)  automated  library  systems  (33%), 
and  e)  camcorders  (32%) . 

A  relevant  finding  of  the  study  was  that 
televisions  and  videocassette  recorders  were  seldom 
requested  in  the  five  year  plan.    This  phenomena  can  be 
explained  by  the  fact  that  the  state  of  Kentucky  has 
had  a  strong  educational  television  network  for  several 
years  and,  for  a  period  of  time,  offered  matching  funds 
for  the  purchase  of  these  items. 

Two  other  findings  of  note  were:  a)  multimedia 
stations  were  not  reported  to  exist  in  any  library 
media  centers,  and  were  included  in  only  13.6%  of  the 
technologies  plans;  and  b)     only  16%  of  the  library 
media  centers  stated  that  they  have  automated  library 
systems.    However,  only  33%  included  automated  library 
systems  in  their  five-year  technology  plans. 

Perceived  Role  of  the  Library  Media  Specialist 
Within  the  survey,  only  42%  of  the  library  media 
specialists  felt  that  the  Kentucky  Educational  Reform 
Act  (KERA)  along  with  the  Kentucky  Educational 
Technology  System  (KETS)  will  have  an  impact  on  their 
roles  as  library  media  specialist!.     Sixty  percent  of 
the  schools  reported  having  a  technology  committee.  Of 
the  schools  having  a  technology  committee,  the  survey 
indicated  that  60%  of  the  library  media  specialists 
were  involved  in  some  capacity  with  the  technology 
committee.    Only,  12%  of  those  involved  served  as  the 

chair.  ...  , 

Two  findings  were  significant.     Only,  two-thirds 
of  the  library  media  specialists  felt  that  technology 
will  affect  their  collection  development.     In  the 
teaching  of  search  strategies,  28%  of  the  respondents 
stated  they  will  teach  only  the  automated  catalog.  In 
contrast,  sixty-two  percent  stated  that  they  will  teach 
both  the  traditional  card  catalog  and  automated  catalog 
search  strategies. 

The  results  of  further  in  depth  interviews  with 
library  media  specialists  suggested  that  they  feel 
dramatically  unprepared  to  use  technology  themselves  or 
to  help  the  classroom  teacher  integrate  the  technology 
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into  the  curriculum.    Apparently,  the  library  media 
specialists  are  overwhelmed  with  the  variety  of 
technologies  available  and  appear  to  be  confused  in 
determining  the  direction  needed  for  the  library  media 
centers.    Several  library  media  specialists  indicated 
that  they  prefer  to  remain  in  the  traditional  role  of 
the  librarian  where  they  only  care  for  the  books  and 
students  who  come  to  the  library.     Finally,  the 
emerging  role  of  the  library  media  specialists  as  an 
information  broker  is  retrieved  from  a  student 
workstation  not  necessarily  in  the  library  media 
center. 


Summary 

This  survey  provides  a  first  step  in  tracking  the 
technological  changes  as  a  result  of  KERA  and  KETS. 
The  results  of  the  survey  indicated  that  few  library 
media  specialists  are  aware  of  the  newer  technologies 
and  their  role  in  the  library  media  centers. 
Furthermore,  library  media  specialists  continue  to  view 
the  library  as  a  storage  and  lending  area  of  books 
rather  than  a  technologically  rich  information  or 
production  area  for  teachers  and  students. 
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INTERACTIVITY  AND  MULTIMEDIA  INSTRUCTION:  CRUCIAL 
ATTRIBUTES  POR  DESIGN  AND  UTILIZATION, 


Increasing  interest  in  currently  available  multimedia  technologies 
capable  of  providing  interactive  learning  experiences  has  resulted  in  widespread 
enthusiasm.  This  has  been  particularly  true  of  technologies  incorporating  the 
laser  videodisc  Interfaced  with  a  computer  and  other  optical  disc  technologies, 
including  configurations  commonly  identified  as  multimedia  systems. 
Electronic  multimedia  does  hold  the  promise  of  providing  easy  access  to 
information  that  has  untold  richness  of  multiple  images  and  sounds  (Ambron, 
1988).  Multimedia  certainly  provides  the  ability  to  Illustrate  ideas  with 
pictorial,  audio,  text,  or  any  combination  of  stimuli. 

On  the  other  hand,  we  continue  to  read  claims  that  research  suggests  no 
learning  benefits  can  be  gained  by  employing  a  specific  medium  to  deliver 
instruction,  regardless  of  the  learning  task,  learner  traits,  symbolic  elements, 
curriculum  content,  or  setting.  (Clark.  1983;  Clark  &  Salomon,  1987;  Clark  & 
Sugrue,  1988). 

When  the  psychological  effects  of  a  presentational  medium  are 
considered  in  terms  of  the  contribution  they  make  to  specific  educational 
outcomes,  the  concern  is  with  the  effects  of  specific  media  characteristics  on 
specific  individuals  and  with  the  functions  they  accomplish  relative  to  given 
instructional  tasks  (Salomon.  1974).  Recently  the  assumption  that  there  is  a 
one-to-one  correspondence  between  coding  elements  and  afforded  activities,  on 
the  one  hand,  and  specific  modes  of  mental  representation  on  the  other  has  been 
questioned  (Clark  &  Salomon.  1987).  Still,  a  useful  classification  distinguishes 
structural  media  attributes  from  functional  media  attributes  to  facilitate 
differentiating  media  on  the  basis  of  their  function  for  influencing  and 
activating  different  kinds  of  learning  processes  (Heidt,  1977, 1978). 

The  goal  of  this  paper  is  to  explore  attributes  of  multimedia  learning 
resources:  the  physical  resources;  the  learners;  and  instructional  design  factors 
which  appear  to  be  crucial  in  determining  interactive  capability  and  potential 
for  instruction. 


Three  significant  dimensions  will  be  examined,  beginning  with  a 
Sensory  Dimension,  which  includes  structural  media  attributes  and  also 
incorporates  an  Imposed  communication  variable,  message  treatment.  Next  a 
FtoceMlng  Dimension,  which  relates  more  closely  to  the  functional  attributes 
ofmedia.  This  dimension  will  be  approached  primarily  from  the  perspective  of 
what  the  learner  brings  to  the  media.  Finally,  a  Control  Dimension  is 
introduced,  represented  by  a  continuum  from  total  control  exercised  by  the 
program  designer  to  the  ability  of  the  learner  to  manage  the  learnlnit  situation 
completely  free  of  program  control. 

In  terms  of  the  internal  operations  of  the  learner,  we  need  to  consider 
how  the  structural  attributes  of  a  medium  used  as  a  technology  for  instruction 
can  be  utilized  as  functional  attributes  which  facilitate  effective  mental 
processing.  Based  on  the  assumption  that  the  factors  which  underlie  the  use  of 
media  for  instruction  are  the  symbol  system,  the  message,  the  learner,  and  the 
educational  task  (Salomon.  1974).  our  Sensoiy  Dimension  relates  to  both  the 
symbol  system  and  the  message.  The  Processing  Dimension  essentially  treats 
the  learner  s  interaction  with  the  symbol  system,  the  message,  and  the  task  The 
Control  Dimension  deals  with  selected  aspects  of  the  task,  although  overlan 
with  the  message  is  obvious.  K 
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Each  of  the  dimensions  represents  a  continuum.  A  cube  diagram  (Figure  1) 
presents  possible  interrelationships  among  the  various  dimensions. 


Processing 
Dimension 

Determining  (Metaoognitions) 
Responding  (Strong  Schemata) 
Mindlessness 

Proceeding  (Weak  Schemata) 
Perception  of  "Reel  Attributes" 
Passive  Processing 


/////. 

//////X)A 
'/////AX. 


Control 
Dimension 


Browsing 
Guided  Learner  Control 
Design  Intervention* 
System  Contra  (Adaptive) 
System  Control  (linear) 


Sensory 
Dimension 


Figure  1.  Crucial  Attributes  of  Interactive  Instruction. 


THE  8ENSORY  DIMENSION 

The  Sensory  Dimension  suggests  the  possibility  that  as  each  additional 
structural  media  attribute  or  communication  treatment  variable  is  utilized,  the 
potential  for  perceptual  saliency  is  increased.  Salomon's  conclusions  that  it  Is 
the  symbol  system  rather  than  the  technology  of  transmission  which  is  crucial 
for  instruction,  and  that  symbol  systems  are  the  primary,  most  essential 
attributes  of  media  (1974. 1979),  provided  a  point  of  departure  from  treating 
media  as  global,  holistic  entities.  However,  other  aspects  of  symbol  system 
theory  may  be  less  secure.  In  particular  the  assumption  "that  cognitive 
representations  and  processing  are  carried  out  in  various  symbolic  modes  that 
are  influenced  by  the  symbol  systems  employed  by  media,  that  some  of  these 
cognitions  are  unique  counterparts  of  communlcational  symbol  systems,  and 
thus  can  be  cultivated  by  symbol  systems"  (Clark  &  Salomon,  1987,  p.  469).  has 
been  challenged. 

For  the  purpose  of  examining  the  attributes  of  instructional  systems,  the 
concept  of  symbol  systems  is  useful.  Based  on  Goodman  (1968)  an  important 
means  of  analysis  is  notationality.  A  notational  system  (e.g.  languages;  musical 
notation)  consists  of  a  set  of  separate,  discontinuous  characters  correlated  with 
a  field  of  reference  which  is  further  segregated  so  that  any  character  in  the 
system  isolates  the  object  or  objects  it  stands  for.  In  contrast,  nonnotational 
systems  (e.  g.  pictures)  are  continuous  and  unsegregated  with  no  set  of  isolated 
characters  (Gardner,  Howard  &  Perkins.  1974). 
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A  related  aspect  of  the  Sensory  Dimension  has  to  do  with  perceptions  of 
reality  in  the  light  of  potentially  vicarious  mediated  experiences.  Several 
researchers  have  studied  the  perceptions  of  reality  of  television  vlewem.  Dorr 
(19&3)  suggests  various  criteria  which  people  use  to  Judge  reality:  reality  can 
represent  a  simulated  experience  in  which  character*,  actions,  messages,  or 
themes  in  some  way  conform  to  real  life;  something  la  Judged  real  If  it  is  deemed 
possible;  and  a  presentation  is  only  judged  real  if  it  Is  considered  probable  or 
representative  of  reality.  Considered  from  the  viewpoint  of  the 
phenomenological  school  of  philosophy,  perceived  reality  can  evolve  through 
"bracketing"  or  the  willing,  but  temporary,  suspension  of  disbelief.  Hie  physical 
world  isn't  eliminated,  but  people  simply  disengage  from  it  by  bracketing  it  out 
and  accepting  the  content  and  context  of  the  media  as  real  (Jonasaen,  1084). 

A  classification  placing  symbolic  modalities  into  categories  of  digital 
signs  and  iconic  signs  (Levie,  1978)  will  be  familiar  to  most  readers.  As  noted  by 
Knowlton  (1966).  digital  signs  are  essentially  arbitrary  and  do  not  have  natural 
referents,  while  iconic  signs  tend  to  bear  a  concrete  resemblance  to  something 
for  which  they  stand. 

Digital  Codes 

Digital  signs  should  be  considered  in  terms  of  sensory  modality. 
Common  examples  of  digital  signs  in  the  visual  modality  are  printed  words, 
numbers,  and  semaphore  code.  Spoken  words  represent  by  far  the  most 
common  auditory  form  of  digital  signs,  although  Morse  code  and  alarms  of 
various  types  constitute  other  examples  (Levie.  1978). 

Of  much  interest  to  our  search  for  crucial  interconnections  among 
attributes  of  interactive  instruction  is  the  concept  that  human  information 
processing  involves  the  verbal  symbolic  system  in  interaction  with  an  imaginal 
symbolic  system  (Paivio,  1971).  'Thus,  language  can  activate  representations  of 
nonveibal  objects  and  events,  which  may  be  experienced  in  the  form  of  mental 
imagery.  Conversely,  nonverbal  objects  and  events,  or  mental  images,  can  be 
described"  (Paivio,  1983.  p.  206). 

ifflfilc  Codes 

The  human  imaginal  system  Is  specialized  for  processing  spatial, 
simultaneous  information  of  a  relatively  concrete  nature.  The  system  is  also 
considered  to  perform  independent  operations,  called  "mental  imagery",  or 
'Visual  thinking".  Iconic  signs  in  the  visual  modality  include  pictures,  statues, 
and  gestures.  Iconic  signs  in  the  auditory  modality  include  sound  effects  and 
music  (Levie,  1978).  Although  these  symbol  systems  are  low  in  notationality, 
they  should  not  be  considered  easier  nor  more  difficult  to  comprehend  or  learn 
than  other  systems  (Gardner,  et  al.  1974;  Clark  &  Salomon,  1987). 

Iconic  -  Motion  &  Audio 

Formal  attributes  of  audiovisual  presentations  include  action,  pace, 
visual  techniques,  and  verbal  and  nonverbal  auditoiy  events.  Such  attributes 
result  from  production  and  editing  techniques  and  are  defined  independently  of 
content.(Huston  &  Wright.  1983). 

Several  studies  have  analyzed  the  relationship  between  formal 
attributes  and  visual  attention.  Although  many  of  the  studies  involved 
children,  the  results  are  interesting  in  terms  of  their  potential  relevance  for 
learners  of  all  ages.  Production  features  which  tend  to  maintain  and  elicit  the 
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attention  of  young  children  Include  motion,  audio  change,  and  sound  effects 
(Anderson  &  Lorch,  1983). 

Formal  features  of  television  which  possess  the  quality  of  perceptual 
salience  were  identified  by  Huston  and  Wright  (1983).  Attributes  identified  as 
salient  included  physical  activity  of  characters,  rapid  pace,  variability  of 
scenes,  visual  special  effects,  loud  music,  sound  effects  and  peculiar  or  non- 
human  voices.  Features  Identified  as  lacking  perceptual  salience  include 
human  speech,  physical  inactivity,  and  background  music.  Moderate  action 
and  zooms  were  classified  as  having  moderate  perceptual  salience. 

The  effect  of  formal  attributes  is  Influenced  by  the  learner's  greater 
familiarity  with  a  medium's  attributes,  and  the  learner's  cognitive  skills, 
linguistic  and  imaglnal  competence,  and  cultural  knowledge.  Huston  and 
Wright  (1983)  discovered  that  perceptually  salient  attributes  have  much  less 
influence  on  the  attention  and  comprehension  of  older  children  and  adults, 
compared  with  younger  children. 

Treatment:  Entertainment;  Informative;  Bmpathetic 

Cognitive  social  psychology  suggests  that  people  actively  affect  their 
environment  through  their  personal  and  socially  shared  perceptions  and  they 
experience  the  consequences  of  their  activity  in  a  reciprocal,  rather  than  linear 
fashion.  'Thus,  from  the  social  learning  perspective,  psychological  functioning 
involves  a  continuous  reciprocal  interaction  between  behavioral,  cognitive,  and 
environmental  influences"  (Bandura.  1978.  p.  345).  This  reciprocal  paradigm 
seems  to  imply  a  limited  role  for  perceptually  salient  media  attributes, 
however,  communicational  treatment  provides  one  possible  area  of  influence. 

The  concept  of  treatment  relates  to  the  manner  in  which  the  content, 
code,  and  structure  of  an  instructional  system  is  organized  and  presented.  One 
classification  of  treatment  forms  identifies:  "expository";  "dramatic";  and 
"personal  involvement"  (Kemp  &  Smellie,  1989). 

Treatment  can  also  be  considered  an  aspect  of  communicational  intent. 
It  is  possible  to  distinguish  between  different  kinds  of  communicational  events 
according  to  their  perceived  intent:  intent  to  entertain;  intent  to  convey  (inform 
or  share);  and  intent  to  change  (Salomon,  1981). 

Familiarity  with  the  formal  attributes  of  popular  media  could  be  a 
possible  recommendation  for  using  an  entertainment  treatment  for  instruction. 
Research  results  suggest  otherwise.  For  example,  Ksobiech  (1978)  found  that 
university  undergraduates  who  were  told  that  a  televised  lesson  would  be 
entertaining  performed  poorly  on  a  subsequent  exam,  compared  to  a  group  who 
were  told  they  would  take  a  test  following  the  presentation,  and  another  group 
which  was  told  they  were  to  evaluate  the  lesson. 

Messages  with  treatments  which  convey  the  perception  that  the 
communicator  is  definitely  attempting  to  influence  or  persuade  the  receiver  will 
tend  to  be  seen  to  limit  one's  freedom  of  thought  or  choice,  and  consequently  are 
likely  to  be  countereffective.  Events  which  convey  the  perception  that  the  only 
intent  is  to  convey,  or  share  information  are  considered  more  effective 
(Salomon,  1981).  However,  potential  problems  with  excessive  use  of  sterile 
information  treatments  could  exist.  Informative  and  interesting  would  seem  to 
be  a  minimal  guideline,  calling  upon  the  creative  talents  of  instructional 
designers.  A  possible  approach  involves  dramatic  treatments  which  convey  the 
communication  intent  of  conveying  information. 


ERLC 


601 


Some  of  the  communlcational  intent  of  designers,  even  when  the 
principal  goal  is  to  inform,  includes  persuading  (motivating)  the  learner  to 
accept  the  message  as  well  as  processing  the  Information,  The  personal 
involvement  treatment  could  counteract  negative  effects,  but  Is  difficult  to  cany 
out.  An  area  of  possible  exploration  would  be  to  attempt  what  might  be  called  an 
empathetic  treatment.  The  goal  would  be  to  demonstrate  the  personal  value  of 
the  communication  to  the  receiver,  in  order  that  tho>  attribution  of  intent  would 
not  be  perceived  as  promoting  the  vested  interests  ol  the  communicator. 

THE  PROCESSING  DIMENSION 

A  popular  assumption  suggests  that  external  forces  strongly  influence  an 
individuals  knowledge  acquisition,  attitudes,  behaviors,  and  cognitive  skill 
mastery.  Learners  are  considered  to  be  reactively  controlled  by  media  and 
instructional  displays  and  the  learner  is  perceived  as  an  active  responder  to  the 
medium's  demands.  The  medium  can  be  said  to  differentially  activate  cognitive 
or  emotional  responses  from  individuals  with  particular  cognitive  abilities  or 
tendencies.  The  nature  of  the  responses  depends  jointly  on  the  attributes  of  the 
medium  and  on  those  of  the  individual  (Salomon,  1983;  Anderson  &  Lorch, 


An  alternative  view  contends  that  attention  to  instructional  displays  is 
actively  under  the  control  of  the  learner.  The  learner  is  perceived  not  only  as  a 
responder,  but  also  as  a  potential  determiner  of  the  experience.  Learners  do  not 
necessarily  respond  to  the  "real"  attributes  of  the  medium,  but  apply  their  own 
often  culturally  shared  perceptions  and  attributions  (the  partial  results  of  prior 
exposure  to  the  media),  which  in  turn  affect  the  kind  of  experiences  the  learner 
realizes  (Salomon,  1983;  Anderson  &  Lorch,  1983). 

There  is  reason  to  believe  that  the  two  perspectives  are  complementary 
People  are  affected  by  their  environments,  but  they  actively  Influence  their 
environments  through  their  personal  and  socially  shared  perceptions.  This 
reciprocal  viewpoint  implies  that  efforts  to  account  for  the  impact  of 
multimedia  instructional  systems  should  consider  what  the  leamer  brings  to 
the  setting  at  least  as  much  as  what  the  system  presents  to  the  learner  (Anderson 
&  Lorch,  1983). 

The  initial  stages  on  our  continuum  of  processing:  Perception  of  "Real 
Attributes'  and  Processing  (Weak  Schemata)  basically  represent  a  traditional 
S-O-R  model  of  learning  where  the  "O"  represents  internal  processing  which 
links  the  stimulus  and  the  response.  Here  attention  and  reaction  to  the 
environment  and  its  displays  is  dominant.  The  stages:  Responding  (Strong 
Schemata)  and  Determining  (Metacognitions)  represent  a  cognitive  processing 
paradigm  where  previously  developed  schemata  incorporating  previous  skill 
information,  and  knowledge  guide  the  learner's  perception  and  largely 
determine  present  and  future  interactions.  The  stages:  Passive  Processing  and 
Mindlessness  represent  aspects  of  each  paradigm  where  superficial,  automatic 
or  an  absence  of  processing  takes  place. 

It  is  suggested  that  both  stages  of  the  processing  continuum  operate  when 
a  learner  encounters  instructional  displays  and  that  they  usually  act  in  concert 
(Salomon,  1983).  Top-down  processing  is  guided  by  anticipatoiy  schemata 
typical  of  search  behavior  ,  %vhile  bottom  up  processing  is  guided  mainly  by  the 
saUency  of  stimulus  properties.  "Apparently,  the  more  one  knows  about  a  topic, 
and  the  better  a  new  unit  of  material  fits  into  a  preexisting  frame,  conception,  or 
schema,  the  more  top-down  processes  dominate  (assimilation);  when  the 
material  is  relatively  novel  (but  not  too  novel)  and  being  handled  by 
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impoverished  schemata,  bottom-up  processes  dominate  (accommodation)" 
(Salomon.  1983.  pp.  185-186) 


Passive  Processing 

Instructional  media  and  instructional  systems  have  often  been 
classified  as  "message  or  stimulus"  oriented  or  as  'Response"  oriented.  Stimulus 
oriented  media  are  characterized  by  their  emphasis  on  input  to  students,  the 
message  and  its*  design,  with  very  little  concern  for  the  explicit  nature  of  the 
response(s)  students  arc  to  make  to  a  particular  message  or  portions  of  it 
(Gropper,  1976).  While  clearly  taking  a  response  oriented  approach.  Cropper 
labeled  media  which  do  not  themselves  impose  the  conditions  thought  to  be 
necessary  for  learning,  such  as  use  of  feedback,  as  passive.  Such  media  as  films, 
television,  radio,  slides,  slide-tape,  audiotape,  and  print  were  identified  as 
passive  carriers.  Active  carriers  (e.g.  programmed  instruction.  CAI) 
deliberately  and  systematically  dictate  particular  conditions  for  learning.  Such 
classifications  may  have  contributed  to  common  perceptions  that  motion 
pictures,  television  and  other  presentation  media  facilitate  only  passive 
processing. 

Termination  of  attention,  the  equivalent  of  passive  processing,  would  be 
expected  to  occur  if  presented  material  were  sufficiently  unfamiliar  or  difficult 
and  the  learner  was  unable  to  activate  and  apply  a  comprehension  schema 
(Anderson  &  Lorch.  1983). 

Perception  of  'Heal  Attributes" 

Processing  of  popular  media  has  been  considered,  if  not  passive,  at  least 
reactive  as  opposed  to  active.  Extrapolation  of  this  approach  to  instruction 
suggests  that  an  audiovisual  medium  elicits  and  maintains  attention  via  salient 
formal  features,  both  visual  and  auditory.  The  direction  of  influence  is  from  the 
medium  to  the  learner.  Hie  influence  of  learner  intentions,  plans,  strategies, 
and  previous  experience  are  minimal.  Relatively  automatic  comprehension  and 
retention  processes  are  assumed  to  occur,  once  attention  has  been  gained 
(Anderson  &  Lorch.  1983). 

Salomon  (1981)  suggests  that  media  can  induce  "naive  changes"  to 
schemata  when  the  schemata  are  weak,  poorly  integrated,  isolated,  not  salient, 
not  important,  or  not  readily  available.  Perception  of  the  content  of  an 
instructional  presentation  may  occur  when  the  salience  of  the  formal  features 
elicits  and  maintains  attention  to  the  system.  This  has  been  equated  with  a  type 
of  "exploration"  as  the  learner  responds  to  immediate,  salient,  discrete  aspects 
of  the  stimulus  setting.  Exploration  is  most  common  when  the  learner  is 
unfamiliar  with  the  situation  being  presented  (Huston  &  Wright.  1983). 

Processing  (Weak  Schemata) 

Exploration  may  be  said  to  move  gradually  to  perceptual  search  when  the 
learner  is  guided  by  internally  generated  goals,  rather  than  by  external  sensory 
events.  At  this  stage  some  of  the  features  that  attract  attention  may  be 
perceptually  salient,  but  learners  are  attracted  to  the  display  features  because  of 
their  informativeness  or  relevance  to  personal  goals,  rather  than  on  their 
perceptual  characteristics  (Huston  &  Wright.  1983). 

Even  though  the  schemata  may  be  well  integrated,  salient,  and 
important  in  the  learner  s  repertoire,  media  can  influence  changes  to  schemata 
which  arc  insufficiently  developed  to  handle  certain  novel  events.  Such  actions 
have  been  characterized  as  information  seeking  processes  (Salomon.  1981). 
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Wndlessness 


Mindlessness,  or  stereotypical  reenactment  represents  the  absence  of 
active  conscious  processing.  It  has  been  shown  that  people  engage  in  mindless 
behavior  when  they  encounter  events  that  appear  to  them  as  highly  familiar, 
overlearned,  and  repetitious.  Individuals  engaged  in  mindless  activity  may  give 
the  appearance  of  mindful  action,  but  new  information  is  not  actually 
processed.  In  particular,  when  the  structure  of  a  communication  is  congruent 
with  a  receivers  past  experience,  it  may  result  in  behavior  which  is  "mindless" 
of  relevant  details.  (Langer,  Blank  &  Chanowitz,  1978).  Thus  what  learners  are 
capable  of  doing  Is  not  necessarily  what  they  actually  do.  The  mindlessness 
phenomena  would  seem  to  account  for  the  performance  of  learners  who  appear 
quite  capable,  but  continually  fall  to  master  curriculum  goals,  despite  repeating 
the  same  material  several  times. 

Responding  (Strong  Schemata) 

It  is  at  this  stage  that  prior  knowledge  and  skills,  which  are  stored  in  the 
schemata  and  are  available  for  processing  come  into  play.  Thus,  the  essence  of 
the  reciprocal  paradigm  which  "postulates  that  personal  dispositions, 
attributions  of  Intent  and  meaning,  communicational  behaviors,  and 
educational  outcomes  are  reciprocally  related  to  each  other.  We  are  influenced 
by  others'  messages,  but  it  is  our  (often  a  priori)  interpretation  of  the  messages 
that  influence  the  way  we  are  Influenced"  (Salomon,  1981,  p.  21 1). 

Determining  (Metacognltions) 

Metacognitiona  tell  us  when,  under  what  conditions,  and  for  what 
purposes  we  are  to  apply  schemata.  Learners  influence  the  way  they  interact 
within  an  instructional  system,  not  just  by  responding  through  the  skills  and 
knowledge  the  system  evokes  in  them,  but  also  through  metacognltions  that 
they  apply  to  it.  Learners  may  also  develop  new  internal  strategies  about  how  to 
learn  as  they  interact  with  an  Instructional  system.  At  this  stage  the  learner  is 
determining  not  only  what  material  will  be  processed,  but  also  how  it  will  be 
processed.  We  could  expect  learners  to  apply  alternative  schemata,  if  available, 
depending  upon  their  Intentions,  goals,  and  the  perceived  demands  of  the  setting 
(Salomon.  1981;  1983).  * 


THE  CONTROL  DIMENSION 

A  crucial  aspect  of  interactive  Instruction  which  relates  to  the  learner's 
processing  of  the  media  within  a  complex  instructional  system  is  the  manner  in 
which  the  system  permits  control  during  learning.  Under  learner  control 
adaptive  instructional  decisions  are  made  by  the  student,  in  contrast  to  system 
control  where  instructional  decisions  (adaptive  or  linear)  are  made  for  the 
student.  Design  decisions  along  this  dimension.  In  interplay  with  the  learner 
and  the  medium  would  seem  to  determine  the  true  nature  and  quality  of  the 
interaction  involved. 

If  we  accept  the  principles  of  interaction  Implied  in  the  "Reciprocal 
Paragigm"  and  current  claims  of  cognitive  learning  theoiy:  constructlvistic 
(Bednar.  Cunningham,  Duffy.  &  Peny.  1991;  Dully &Jonassen,  1991)  and 
generative  (Wittrock.  1977;  1979).  it  seems  evident  that  even  in  situations  in 
which  the  presentations  are  selected,  sequenced  .  and  paced  for  the  student  a 
great  deal  of  learner  control  will  still  be  exercised  by  the  student.  Concluding 
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that  all  instruction  involves  some  learner  control,  Merrill  (1984)  suggested  that 
"the  challenge  is  not  whether  or  not  learner-control  should  be  made  available, 
but  rather  how  to  maximize  the  student's  ability  to  use  the  learner  control 
available  (1984,  p.  239).  There  is  little  positive  evidence  favoring  total,  unaided 
learner  control.  Research  indicates  that  learner  control  with  some  form  of 
coaching  has  been  consistently  superior  to  unassisted  learner  control 
(Hannafin.  1985).  Thus,  'The  capabilities  of  the  learners  to  use  control  options 
effectively  is  a  more  salient  consideration  than  the  capability  of  the  technology 
to  permit  control"  (Hannafin,  1985.  p.  243). 

Perhaps  predictably,  the  research  has  begun  to  suggest  that  learner 
control  is  not  a  unitary  construct,  but  rather  a  collection  of  atrstegi et  that 
function  in  different  ways  depending  on  what  la  being  controlled  by  whom" 
(Ross  &  Morrison,  1989,  p.  28),  Thus,  a  continuum  of  instructional  control  Is 
implied  (Figure  2),  and  indeed  the  research  continues  to  feature  attempts  to 
identify  specific  situations  in  which  learner  control  or  program  control,  or 
some  combination  along  a  continuum,  is  most  effective. 

An  Instructional  Control  Continuum 

— i — i — i— i — 

System      System  Design       Guided  Browsing 

Control  Control  Interventions  Learner 
(linear)   (Adaptive)  Control 

Figure  2.  Representation  of  a  Continuum  of  Instructional  Control. 


System  Control  (linear) 

System  control  of  instruction  is  characterized  as  external,  and  is 
identified  with  instruction  in  which  all  learners  follow  a  predetermined  route 
established  by  the  designer  without  using  individual  Judgement  as  to  the 
appropriateness  of  the  path  (Hannafin,  1984).  Externally  controlled  CAI  has 
proven  effective  in  drill  and  practice  tasks  Involving  lower  order  intellectual 
skills  (Hannafin,  1984).  however  the  evidence  suggests  that  contextual  and 
substantive  information  and  higher  order  skills  may  be  best  taught  using 
learner  control  (Kulik.  Bangert.  &  Williams.  1983). 

Syitem  Control  (Adaptive) 

Although  some  CAI  and  Interactive  Video  instruction  has  been  designed 
along  the  lines  of  a  linear  programming  model,  system  control  does  not 
necessarily  connote  a  linear  approach.  Tutorial  CAI  is  more  the  ideal.  An 
example  of  rigid  system  control  is  a  version  of  tutorial  CAI.  based  on  a 
performance  or  response  driven  adaptive  instructional  model.  The  learner  is 
given  practice  problems  as  part  of  the  presentation  and  the  next  display  is 
determined  from  the  response.  Any  adaptation  is  the  same  for  all  learners,  but 
if  some  branching  is  used,  a  learner  would  see  one  display  if  the  response  to  the 
previous  frame  was  correct,  and  a  different  one  if  it  was  incorrect.  It  has  been 
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suggested  that  provided  the  content  Is  fixed,  such  a  system  Is  similar  to  a  lecture, 
and  even  more  limited  than  a  textbook  in  providing  the  opportunity  for  the 
learner  to  override  the  system  .  Even  systems  which  Incorporate  an 
idiosyncratic  adaptive  model  for  each  student  have  been  programmed  in  such  a 
way  that  learner  control  is  very  limited  (Merrill,  1984). 

Design  Interventions 

Several  design  enhancements  within  u  system  control  approach  are 
supported  by  research  evidence.  Learning  has  been  shown  to  be  most  effective 
when  criterion  questions  are  embedded  throughout  the  lesson.  As  well, 
questioning  and  response  feedback  procedures  have  been  shown  ia  increase 
comprehension  and  attention  to  instruction  using  interactive  video  (Hannafln 
1985).  Providing  examples  in  a  variety  of  contexts  has  been  identified  as  a 
promising  strategy  which  facilitates  independent  adaptive  decision  making  bv 
learners  (Ross,  Morrison  &  O'Dell,  1989). 

Idiosyncratic  adaptive  instruction,  and  instruction  utilizing  the  abeve 
design  enhancements  may  each  be  seen  as  an  attempt  to  assist  the  individual  *o 
most  successfully  achieve  desired  program  goals  and  objectives.  Thus  both  may 
be  more  effective  In  situations  where  there  is  clear  agreement  on  the  desirability 
of  all  students  mastering  relatively  specific  instructional  outcomes. 

Guided  Learner  Control 

Situations  in  which  considerable  learner  control  is  afforded,  but  with 
advice  given  to  the  learner  (internal  program  guidance)  is  a  promising  approach. 
Research  evidence  verifies  the  desirability  of  including  some  form  of  "coaching" 
to  assist  learners  in  making  informed  decisions  (Hannafln.  1984;  Ross,  1984; 
Tennyson  and  Buttrey,  1980).  Successful  procedures  have  been  developed  in 
research  settings  which  offer  guidance  upon  which  individual  student's 
decisions  can  be  based.  The  program  can  advise  the  learner  as  to  the  number 
and  types  of  practice  items  or  examples  recommended,  current  masteiy  status, 
arid  other  performance  features.  The  advisement  is  based  upon  the  individuals 
past,  current,  or  cumulative  performance  during  a  given  lesson.  The  learner 
matatains  control  by  being  able  to  accept  or  reject  the  offered  advice  (Hannafln 
1984;  Hannafln,  1985). 

Emerging  learning  theory  suggests  that  even  in  settings  where 
instructional  displays  are  selected  for,  sequenced  for,  and  paced  for  the  student 
a  great  deal  of  learner  control  will  still  be  exercised  by  the  student.  While 
C^lng.in  a  ^sson*  tamers  acquire  internal  strategies  about  how  to  learn,  in 
addition  to  subject  matter.  This  metacognitton  refers  to  the  "how  to  study- 
model  which  the  student  uses  to  guide  interaction  with  the  Instructional  system 
(Merrill,  1984).  According  to  Merrill  "the  challenge  is  not  whether  or  not 
learner-control  should  be  made  available,  but  rather  how  to  maximize  the 
student's  ability  to  use  the  learner-control  available"  (1984,  p.239). 

A  major  thrust  of  research  by  Merrill  and  associates  with  adaptive  CAI 
featuring  internal  learner  control  involves  attempting  to  guide  the  learner  by 
providing  carefully  defined  strategy  ootions  for  controlling  display 
presentation.  They  suggest  that  students  can  be  taught  more  appropriate 
processing  procedures  and  encouraged  to  use  the  strategies  (metacognitions) 
where  appropriate.  A  promising  early  result  was  the  conclusion  regarding  the 
hypothesis:  "Students  who  are  provided  directions  for  conscious  cognitive 
processing  of  the  information  presented  will  perform  better  than  will  students 
who  are  left  to  their  own  internal  processing  strategics".  Students  who  received 


606  ^tu 


directions  about  how  to  process  the  information  scored  higher  on  the  posttest 
than  students  who  didn't  receive  the  directions. 

Browsing 

Browsing  is  the  label  used  to  subsume  the  variety  of  things  which  could 
happen  at  the  end  of  the  continuum  where  the  learner  is  generally  considered  to 
be  in  control,  or  at  least  interacting  directly  with  the  system.  One  attempt  to 
define  browsing  describes  it  as  the  intellectual  process  of  acquiring 
individualistic  knowledge  (Jonassen,  1989).  A  total  learner  control  orientation 
would  imply  allowing  learners  to  make  decisions  about  what  they  want  to  learn 
by  selecting  options  which  are  presented  at  different  points  in  the  lesson  (Ross, 
1984).  The  least  sophisticated  use  of  multimedia  as  an  information  reference 
tool  might  be  termed  nibbling  ,  with  the  user  aimlessly  probing  through  a  dense 
information  base. 

However,  it  is  necessary  to  consider  other  approaches  to  learning,  as  well 
as  other  approaches  to  utilizing  the  information  potential  of  multimedia 
systems. 

Recent  controversy  over  the  instructional  design  model  commonly 
employed  within  the  field  of  instructional  technology  has  implication!  for 
interactive  multimedia.  Claiming  that  instructional  technology  had 
accommodated  cognitive  psychology  in  its  theory,  but  veiy  little  in  its  practice, 
Jonassen  (1990)  urged  the  field  to  move  toward  a  constructtvist  view  of 
instructional  design.  He  pointed  out  that  cognitive  information  processing  is 
not  fundamentally  different  from  behaviorism,  since  both  rely  upon  an 
objectlvistic  conception  of  knowledge. 

A  definition  of  constructivism  provided  by  Jonassen  (1990)  will  be  useful 
to  a  further  discussion  of  the  issue: 

Constructivism  is  the  belief  that  knowledge  is  personally 
constructed  from  internal  representations  by  individuals 
using  their  experiences  as  a  foundation.  Knowledge  is 
based  upon  individual  constructions  that  are  not  tied  to 
any  external  reality,  but  rather  to  the  knower's 
interactions  with  the  external  world.  Reality  is  to  a  degree 
whatever  the  knower  conceives  it  to  be  (p.  32). 

An  extreme  position  on  constructivism  advocates  that  it  become  the  sole 
basis  for  instructional  design.  It  is  claimed  that  since  the  goal  of  instruction  for 
both  behaviorist  applications  and  cognitive  information  processing  is  to 
communicate  or  transfer  knowledge  to  learners  in  the  most  efficient,  efFective 
manner  possible,  it  is  not  tenable  to  add  constructivist  theory  to  the 
"smorgasbord"  of  behaviorism  and  cognitive  information  processing  (Bednar, 
Cunningham,  Duffy  &  Peny.  1991). 

Relgeluth  (1991)  responded  to  Bednar,  et  al  as  well  as  other  writers  who 
suggest  such  an  extreme  view  of  constructivism.  He  pointed  out  that 
constructivism  does  have  much  of  value  to  educators,  but  that  other  perspectives 
do  as  well.  Some  of  the  major  tenets  of  constructivism  have  long  been  espoused 
by  instructional  technology.  For  example  instructional  designers  *have  for 
some  time  advocated  'situating*  learning  experiences  In  authentic  activities* 
(1991,  p.  34).  The  need  to  couch  instruction  In  a  context  that  is  meaningful  to 
learners  has  been  identified  as  perhaps  the  most  significant  aspect  of 
const ructlvistic  design  (Jonassen,  1991).  Certainly  most  instructional 
designers  and  teachers  would  not  disagree,  but  they  might  also  report  much 
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frustration  from  attempts  at  trying  to  situate  their  instruction  in  appropriate 
contexts. 

Recent  interest  in  multimedia  has  been  accompanied  by  claims  and 
suggestions  of  potential  user  empowerment.  We  agree  with  Merrill  (1991).  who 
advocated  that  moderate  constructivism  has  much  that  should  be  considered  by 
instructional  designers,  and  pointed  out  that  much  of  his  own  work  in 
developing  ID2  is  an  attempt  to  provide  tools  that  enable  the  development  of  the 
type  of  learning  environments  that  they  describe.  Merrill  argued,  however  that 
the  assumptions  about  the  learning  process  made  by  extreme  constructivistic 
authors  as  Bednar.  etal  (1991)  are  unnecessarily  restrictive  and  may  actually 
prevent  the  more  effective  instruction  they  advocate. 

Much  of  the  enthusiasm  for  multimedia  is  for  purposes  other  than 
intentionally  created  learning  environments.  It  is  multimedia  systems  which 
are  designed  to  be  used  as  Information  reference  tools  which  actually  represent 
the  focus  of  much  of  the  discussion  relating  to  learner  empowerment  and 
freedom. 

Learners  using  hypermedia  are  said  to  be  freed  from  the  linear  tradition 
oi  printed  text.  "They  are  encouraged  to  browse  a  hyperdocument.  move  easily 
among  vast  quantities  of  information  according  to  plan  or  serendipity,  follow 
relationships  pre-coordinated  by  the  author  or  create  their  own  paths  through 
the  information"  (Marchionini.  1988.  p.  8). 

Freedom  and  empowerment  mean  that  learners,  particularly  where 
specific  instructional  outcomes  are  an  issue,  require  new  strategies  and  tools  for 
making  the  best  use  of  their  time  and  effort.  Hie  challenge  to  designers  is  to 
,  S^f         t0  help  leamers  manage  this  freedom  of  learning  (Marchionini. 
1988).  Thus,  the  importance  of  careful  design  and  development  is  no  less 
critical  in  the  design  of  instructional  hypermedia  environments  than  for  other 
learning  formats.  "In  order  to  successfully  use  a  hypermedia  application 
learners  must  be  provided  with  appropriate  and  clear  navigational  and 
conceptual  tools  in  order  to  explore  even  the  best-designed  systems"  (Morariu. 
IBoo,  p.  19). 

Given  the  power  of  emerging  technologies,  would  It  not  be  possible  for  a 
multimedia  system  to  provide  in  effect  a  continuum  of  experiences?  Could' not 
the  potential  resources  for  structured  and  unstructured  experiences  alike  reside 
within  something  called  an  intelligent  hypermedia  knowledge  system  for 
learning?  And  could  not  such  a  system  accommodate  objective  instruction  In 
well-defined  domains  as  well  as  provide  an  open  environment  for  active 
learning  which  enables  learners  to  make  their  own  decisions 

Most  of  this  discussion  has  dealt  with  inquiry  in  various  areas  of 
education  and  instructional  technology  in  the  belief  that  It  has  the  promise  of 
being  translated  into  practical  guidelines  for  multimedia  development.  Inquiry 
specific  to  multimedia  is  beginning  to  appear  in  the  literature,  much  of  which 
suggests  careful  and  systematic  research  on  specific  aspects  of  hypermedia  For 
example.  Story  and  Harvey  (1991)  reviewed  existing  research  on  hypermedia 
browsers  (structural  iconic  browsers,  or  maps  intended  to  facUitate  navigation 
through  a  hypermedia  system).  However,  much  of  the  literature  deals  wim 
development,  which  as  Park  (1991)  suggests,  essentially  involves  hypertext  and 
not  true  hypermedia  or  multimedia  at  all. 
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EMPOWERING  LEARNERS  THROUGH 
METACOGNITIVE  THINKING,  INSTRUCTION  AND  DESIGN 


by  Ming-Fen  Li 


BACKGROUND 

Since  the  enthusiasm  for  school  restructuring  swept  the  U.S.,  the  role  of  learners  has 
received  increasing  attention.  Learners  are  no  longer  perceived  as  passive  knowledge- 
consumers.  What  interests  educators  now  is  how  to  create  a  learning-centered  environment  to 
enhance  learning  achievement.  With  this  increased  awareness  of  the  importance  of  learners 
numerous  proposals  have  been  brought  forward  to  restructure  public  education  through  ' 
technology,  to  empower  teachers  to  redesign  curricula  and  reorganize  learning  activities  to 
involve  parents  in  extending  learning  beyond  school,  and  to  rearrange  learning  resources  to 
narrow  the  gap  between  societal  needs  and  school  education.  While  much  effort  is  expended  to 
make  learners  the  decision-makers  of  their  own  learning  through  the  above  approaches  I 
believe  that  the  success  of  a  learning-centered  environment  lies  not  merely  in  restructuring 
the  environment  external  to  the  learners,  but  also  in  enabling  learners  to  cultivate  reflection 
upon  their  own  learning. 

Cognitive  psychologists  have  identified  the  way  one  exercises  mental  processes  as  the 
most  significant  determinant  in  successful  learning.  Flavell  (1978)  coined  the  phrase 
"metacognition"  for  the  way  learners  controlled  and  directed  their  own  mental  processes.  Since 
then,  metacognition  has  received  more  and  more  attention  and  has  been  applied  in  these  contexts 
ot  learning.  It  is  important  to  know  how  to  present  metacognition  in  various  contexts  of 
learning,  instruction  and  design  in  order  to  enhance  learners'  mental  capabilities  and  potential 
Teaching  students  how  to  exercise  their  metacognition  means  kindling  their  intrinsic  motivation 
and  enabling  them  to  assume  the  responsibility  of  learning,  instead  of  overrelyina  on  the 
teacher's  external  motivation. 

STATEMENT  OF  THE  PROBLEM 

After  Flavell  coined  the  term  metacognition  in  the  early  1970s,  there  has  been  an 
•"creasing  interest  in  exploring  the  essence  of  the  concept  and  in  constructing  instructional 
models  for  promoting  metacognitlve  learning.  A  great  deal  of  research  (Brown,  Bransford 

C?T°ne'  19831  PariS  *  Lindauer'  1982:  We»™n.  1983  Paris,  Wasik,  &' 
Vander  Westhuizen,  1988;  Pressley,  Borkowski,  &  O'Sullivan,  1985)  supports  the  Importance 
of  metacognition  in  cognitive  development  and  academic  learning,  demonstSing  that  leaders 
who  are  metacognitively  active  tend  to  perform  better  on  cognitive  tasks  than  those  who  are  not 
and  that  metacognition  distinguishes  an  expert  problem  solver  from  a  novice.  Up  tote  present 
2  in  ^cognition  covers  several  primary  areas,  such  as  the  enhancement  of  disabled 
2S2£T  2?'        9  comDrehenslon  and  monitoring,  and  problem  solving.  With  successful 
appncation  in  these  areas  the  role  of  metacognition  is  becoming  prominent.  It  has  received 
n,  I    H!nre    t  00     1)0,11  educatlone"  end  training  contexts,  like  the  other  higher-level 
thinking  skills,  such  as  critical  thinking,  problem-solving  and  inquiry  (Leshin,  Pollock  & 
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Reigeluth,  1992). 

Nevertheless,  despite  the  fruitful  findings  in  the  literature,  scholars  do  not  agree  on  the 
definition  of  metacognition,  defining  it  in  various  ways  in  the  course  of  the  past  decades.  This 
has  caused  a  lot  of  confusion  and  remains  a  thorny  problem  to  be  solved.  While  the  evolution  of 
the  diverse  definitions  of  metacognition  can  be  attributed  to  the  new  discoveries  in  learning 
psychology,  and  scholars1  different  use, of  it  (Jones  &  Idols,  1990),  there  is  a  need  to  borrow 
new  perspectives  in  order  to  come  to  a  better  and  more  comprehensive  definition. 

Current  models  of  metacognitive  instruction  are  predominantly  developed  for  reading 
comprehension  and  science  education.  As  learning  tasks  become  more  and  more  specialized  in 
education  and  training,  these  models  are  inadequate  to  accommodate  the  wide  varieties  of  content. 
Many  instructional  models  on  metacognition  have  simplified  metacognitive  instruction  into  a  set 
of  procedures,  excluding  many  humanistic,  social  and  cultural  factors.  Despite  the  various 
models  of  metacognitive  instruction,  there  Is  a  poverty  of  comprehensive  information  on  how 
practitioners  can  design  their  own  lessons  by  Incorporating  these  models,  since  they  are 
instructional  models  rather  than  instructional  design  models. 

All  of  this  calls  for  a  close  examination  of  the  nature  of  metacognition  and  the  way  people 
teach  it.  Rather  than  teaching  metacognition  as  a  group  of  strategic  skills  encompassing  the 
whole  story  of  a  learning  task,  !  propose  that  the  role  of  metacognition  should  be  repositioned  in 
a  larger  context  of  learning.  It  is  imperative  to  provide  guidance  for  practitioners  on  how  to 
apply  metacognitive  instruction  to  a  wider  range  of  content,  and  design  metacognitive 
instruction  without  sacrificing  the  knowledge  of  how  people  develop  their  metacognition,  or 
neglecting  the  new  ideologies  and  assumptions  which  underlie  a  specific  learning  task  or  content. 

PURPOSE  OF  THE  RESEARCH 

The  purpose  of  the  research  is  to  provide  a  se*  of  perspectives  for  empowering  learners 
through  metacognitive  thinking,  instruction  and  design.  These  perspectives  are  based  on  a  new 
conceptualization  of  metacognition  and  an  integration  of  learning  and  instruction.  Previous 
conceptions  about  metacognition  are  problematic  since  most  of  them  place  the  emphasis  on 
monitoring  and  learning  strategies,  neglecting  the  true  nature  of  metacognition  and  the  process 
by  which  learners  acquire  metacognitive  skills.  Current  instructional  theories  have  been 
overemphasizing  the  optimization  of  instruction,  separating  to  a  large  extent  design  from 
instruction  (Winn,  1989;  Streible,  1991),  making  the  enterprise  of  instructional  design 
difficult  to  apply  to  many  contexts.  In  addition,  because  of  its  focus  on  strategies  acquisition, 
the  existing  design  of  metacognitive  instruction  is  procedural-oriented,  isolating  metacognitive 
learning  from  creativity  and  other  critical  thinking  skills.   Therefore,  this  research  will 
ground  the  design  of  metacognitive  instruction  more  solidly  on  the  nature  of  metacognitive 
learning. 

CONCEPTUAL  FRAMEWORK 

The  philosophical  conceptual  framework  of  the  research  will  be  the  foundation  of  this 
research.  This  conceptual  framework  primarily  comprises  an  investigation  of  the  nature  of 
human  thinking  and  metacognition,  the  power  of  mental  ability,  the  oneness  of  knowing  and 
doing,  and  reflection  upon  the  buried  SELF  in  this  highly  technological  and  scientific  world. 

The  lost  horizon 

Human  beings  have  always  been  driven  to  innovate  technology  in  order  to  improve  life. 
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While  enjoying  the  fruits  of  technology,  and  relying  more  and  more  on  modern  technology,  they 
gradually  neglect  the  inner  self.  The  journey  of  metacognitlve  learning,  instruction  and  design 
is  to  find  that  buried  self  and  restore  our  full  mental  ability. 

Power  of  mental  thinking 

A  learning  environment  is  usually  easy  to  change  if  resources  are  available  and 
technology  is  ready  for  it.  However,  the  rearrangement  of  the  learning  environment  does  not 
always  result  in  the  reconstruction  of  one's  mindset.  When  technology  fails,  we  should  look  for 
the  cause  not  in  technology  itself  but  rather  in  defects  of  human  mental  perspectives. 
Nevertheless,  people  still  tend  to  be  convinced  by  the  power  of  technology,  rather  than  that  of 
their  mental  thinking.  While  educators  align  themselves  with  various  endeavors  in  school 
reform,  they  should  turn  to  the  learners'  internal  world  to  explore  its  potential  in  order  to 
enhance  achievement. 

Oneness  of  knowing  and  action 

The  power  of  inner  thoughts  lies  in  its  determinant  force  of  driving  actions,  especially 
actions  for  decision-making  about  learning  and  life.  Back  in  the  sixteenth  century,  a  Chinese 
neo-Confucianist,  Wang  Yang-Ming  (1472-1529),  devoted  most  of  his  life  to  figuring  out  the 
inter-relationship  between  knowing  and  action.  He  first  examined  two  premises:  knowing  is 
difficult  and  action  Is  easy;  action  is  difficult  and  knowing  is  easy.  Then  he  proposed  a  theory 
that  knowing  Is  action,  i.e.  the  oneness  of  knowing  and  action.  In  a  similar  manner,  several 
scholars  have  recently  called  for  situated  or  situational  learning,  claiming  that  learning  is 
contingent  upon  doing,  and  only  when  learning  is  indexed  in  context  can  learners  truly  benefit 
from  the  knowledge  (Brown,  Collins,  &  Duguid,  1989).  These  scholars  seemingly  emphasize 
that  learning  comes  with  or  after  doing,  and  that  knowledge  can  not  be  separated  from  the 
context  to  which  it  will  be  applied.  Their  argument  for  integrating  learning  and  doing  also 
exemplifies  Wang's  belief,  and  lays  out  the  approach  that  makes  possible  the  fusion  of  knowing 
and  action. 

MEDIATION  OF  EMPOWERMENT  -  METACOGNITION 

What  is  Metacognition? 

As  stated  earlier,  Flaveil  (1978)  is  the  pioneer  of  research  on  metacognition.  He  places 
emphasis  of  metamemory  on  the  learner's  knowledge  about  variables  related  to  the  person,  task, 
and  strategies.  Scarr  and  Zanden  (1984)  define  metacognition  as  awareness  and  understanding 
of  one's  mental  states,  abilities,  memory,  and  processes  of  behavioral  regulation.  Some 
researchers  simplify  the  term  as  mere  knowledge  about  one's  thought  process  or  thinking  about 
thinking  (Glover,  Ronning,  &  Brunning  1990),  while  others  break  it  up  into  several  subskills. 
For  example,  Brown  (1978)  emphasize  aspects  of  executive  cognition  such  as  planning, 
monitoring,  and  revising  one's  thinking;  Gall,  Gall,  Jacobsen  and  Bullock  (1990)  define  the 
components  of  metacognitive  skills  as  knowing  the  learning  process,  selecting  appropriate 
learning  strategies  and  monitoring  how  one's  learning  strategy  is  working.  Beyer  (1987) 
elaborates  the  definition  of  metacognition  by  incorporating  the  ability  of  planning  how  to  carry 
out  the  task  and  carrying  it  out,  monitoring  one's  process,  adjusting  one's  actions  to  the  plan, 
and  revising  both  plan  and  actions  in  the  process.  Paris  and  Winograd  (1990)  characterize 
metacognition  as  knowledge  about  cognitive  states  and  abilities  that  can  be  shared  among 
individuals.  According  to  them,  metacognitive  skills  include  both  cognitive  and  affective 
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characteristics  of  thinking. 


Function  of  metacoonition 

The  many  definitions  of  metacognition,  however  different  from  one  another,  can  yet  he 
seen  as  a  combination  of  learning  and  monitoring  strategies  (Bonner,  1988).  Most  researchers 
have  now  blended  those  twin  approaches  into  a  definition  that  emphasizes  (a)  '.nowledge  about 
cognitive  states  and  processes  and  (b)  control  of  executive  aspects  of  metacognition  (Borkowski, 
1985;  Brown  et  al.,  1983;  Wellman,  1985;  cited  from  Paris  &  Winograd,  1990).   However,  I 
believe  that  the  core  of  the  concept  is  the  dynamic  thinking  process-critical  and  conscious 
action,  which  is  actually  Wang's  argument  that  knowing  is  doing  from  the  viewpoint  of  learners, 
instead  of  the  proceduralizing  of  the  components  of  metacognition  applied  by  earlier 
researchers,  I  contend  that  looking  at  the  core  of  metacognition,  focusing  on  its  power,  and 
contemplating  how  one  becomes  metacognitive  are  what  educators  need  to  do.  The  dynamic 
reflective  thinking  process  exerts  influence  beyond  controlling  and  directing  one's  learning 
strategies.  It  is  a  key  to  one's  inner  "whole",  a  compass  on  the  sea.  Therefore,  instead  of 
teaching  students  what  to  learn,  teachers  should  have  a  full  understanding  of  the  complex  nature 
of  metacognition  in  order  to  guide  students  how  to  learn  and  think. 

Metacognitive  thinking  functions  to  liberate  learners  from  the  rigid  and  proceduralized 
instructional  environment  imposed  by  the  instructors,  administrators  or  the  whole  school 
system.  It  emancipates  one's  potential,  making  possible  independent,  life-long  learning.  When 
school  reformers  claim  to  create  a  learning-centered  environment  in  schools  and  to  give  more 
freedom  to  learners,  they  should  contemplate  how  to  enable  learners  to  use  such  freedom. 
Therefore,  unlike  most  people  who  concentrate  on  the  components  of  metacognition,  I  would  like 
to  emphasize  its  process  in  this  paper,  recognizing  that  metacognition  is  a  developmental 
process,  and  everyone's  metacognitive  process  is  different. 

NEW  PERSPECTIVES  OF  METACOGNITIVE  THINKING 

HflnnnnftDtuali7atinn  of  matacoanition 

Traditionally,  metacognition  has  been  treated  as  the  combination  of  learning  strategies 
and  monitoring  strategies,  or  the  evaluation  and  revision  of  them.  While  learning  strategies, 
the  newly  learned  in  particular,  tend  to  be  external  to  the  learners,  monitoring,  evaluation  and 
revision  are  learners'  exercise  of  internal  mental  process  on  these  external,  or  to-be- 
acquired,  strategies.  However,  metacognition  cannot  be  studied  in  isolation  from  critical 
thinking  or  creativity,  nor  should  it  be  studied  separate  from  drifting  thoughts.  As  Borkowski, 
Carr,  Rellinger  and  Pressley  (1990)  contend,  metacognition  tends  to  last  for  short  period  of 
time.  How  it  is  interfered  with  by  drifting  thoughts  and  how  it  interacts  with  other  mental 
activities  should  be  clarified  in  order  to  provide  practitioners  with  more  Illuminative  guidance. 

Nature  of  thoughts:  While  characterizing  thinking  by  cognition  and  metacognition, 
Beyer  (1987)  does  not  tell  us  precisely  how  people  really  think.  Scarr  and  Zanden  (1984) 
affirm  that  people  think  in  two  distinct  ways-directed  thinking  and  nondirected  thinking. 
According  to  them,  directed  thinking  Is  a  deliberate,  purposeful,  systematic  and  logical  attempl 
to  reach  a  specific  goal,  whereas  nondirected  thinking  consists  of  a  free  flow  of  thoughts  through 
the  mind,  with  no  particular  goal  or  plan,  and  depends  more  on  images.  The  latter  Is  usually 
rich  in  imagery  and  feelings.  Daydreams,  fantasies,  and  reveries  are  typical  examples.  Peoplo 
often  engage  In  nondirected  thought  when  they  are  relaxing  or  trying  to  escape  from  boredom  or 


worry.  This  kind  of  thinking  may  provide  unexpected  insights  into  one's  goals  and  beliefs. 

Nature  of  metacognition:  Thoughts  are  like  waves  that  surge  all  the  time,  even  when 
we  fall  into  deep  sleep.  They  are  like  wild  monkeys  and  horses;  the  harder  you  try  to  tame 
them,  the  wilder  they  become.  Without  diving  into  the  depths,  we  will  never  feel  the  force  with 
which  they  drive  us.  Tomio  Hlrai  (1978)  says: 

"The  human  mind  is  rarely  as  calm  as  an  untroubled  body  of  water.  At  almost  all  times, 
ripples  or  waves  of  pleasant  or  unpleasant  emotion  are  disturbing  its  tranquility. 
Sometimes,  the  frequency  of  these  disturbances  reaches  such  a  peak  that  the  mind 
resembles  a  raging  sea  in  which  there  is  never  an  ebb  tide  to  bring  calm."  (p.76) 
It  Is  essential  that  educators  identify  the  existence  of  drifting  thoughts  and  the 
everlasting  ebb  and  flow  characteristic  of  the  human  mind.  Unlike  those  who  concentrate  their 
instruction  exclusively  on  metacognitive  skills  and  proceduralize  this  instruction,  I  believe  the 
design,  instruction  and  learning  of  metacognition  should  incorporate  nondirected  thinking  and 
creative  thinking.  The  question  about  whether  drifting  thoughts  interfere  with  metacognitive 
thinking,  or  whether  metacognition  can  potentially  impede  creative  thinking  is  worth 
contemplating. 

Metacognition  and  creativity:  The  essence  of  metacognitive  thinking  is  more 
complex  than  the  various  versions  that  have  been  discussed  above.  Since  metacognitive  thinking 
tends  to  occur  In  short  fits  (Borkowski  et  al,  1990),  when  lea'rners  first  learn  to  use 
metacognitive  thinking,  they  are  like  novice  surfers  who  strive  to  balance  themselves  in  the 
irresistible  and  unpredictable  surging  currents  surrounding  them.  Their  balancing  skills  have 
to  be  developed  in  the  raging  surges.  Similarly,  one's  metacognitive  thinking  has  to  be  developed 
along  with  drifting  thoughts.  Rather  than  dealing  with  metacognition  as  a  mental  activity 
separate  from  others,  I  would  emphasize  that  we  cultivate  metacognition  not  to  suppress 
drifting  thought,  but  to  skillfully  direct  drifting  thoughts  to  metacognition,  where  creativity 
can  be  fostered.  However,  current  literature  does  not  mention  how  one's  metacognition 
interrelates  with  creativity. 

Although  Perkins  (1990)  argues  that  there  is  room  for  mindfulness  and  control  to  play 
in  creative  thinking,  and  that  creative  thinking  can  steadily  become  more  metacognitive. 
changing  in  character  as  it  does  so,  he  does  not  take  into  consideration  the  relationship  between 
drifting  thoughts  and  directed  thoughts.  What  he  proposes  is  more  a  matter  of  integrating  the 
strategic  feature  of  metacognition  with  creative  thinking,  namely  of  extracting  the  strategies 
for  creative  thinking  and  making  them  executive  and  manageable.  The  complex  interplay  of  the 
two  mental  activities  are  not  really  investigated. 

Metacognition  and  habitual  thinking  &  acting  patterns:    Habitual  thinking  or 
acting  is  another  variable  that  competes  with  the  cultivation  or  development  of  metacognition. 
Sometimes,  it  even  plays  a  more  dominant  role  than  drifting  thoughts  in  directing  one's  thinking 
and  acting.  Very  often  people  fail  to  use  metacognition  not  because  they  don't  know  the  value  of 
it,  but  because  they  are  not  able  to  overcome  the  driving  force  of  their  habitual,  routinized  way 
of  thinking  and  acting.  It  is  through  critical  awareness  of  the  dominant  role  of  these  thinking 
and  acting  patterns  that  people  can  rise  up  from  the  bondage  of  these  patterns  and  from  the 
frustration,  anxiety  or  lack  of  confidence  caused  by  the  unsuccessful  adoption  of  new  ways  of 
thinking  and  acting.  Then  they  can  extend  their  visions  to  a  broader  spectrum,  and  the 
expansion  of  one's  repertoire  becomes  possible. 
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Matacoanition  as  critical  awareness  and  conscious  action.  Although 
awarenesf  of  on"  cogni^  states  was  identified  as  a  major  component  of  metacognltlon 
mZm  1978  Scarr  &  Zanden  1984),  the  complexity  of  the  concept  has  not  be«P 
Ei      the eratur  of  mlSLoonWon.  At  best,  it  is  understood  as  the  knowing  o  one's 
memafs afe   learning  process  or  strategies,  behavior  regulation  abHjty  J^-Wj^ 

and  oerceDtion  of  differences  between  self  and  others  in  terms  of  the  internal  charactens^  or 
22?  It  i important  to  highlight  awareness  from  the  two  dimensions  and  assist  learners 
JSL  ^TSr  interna?  processes  Interact  with  the  ^^^^JE^ 
manner,  and  how  such  interaction  might  be  Improved  in  order  to  cultivate  learners  awareness 
at  a  more  conscious  and  critical  level. 

Metacognition  and  reflection:   In  order  to  further  understand  the  nature  of 
metacoanition  it  Is  Important  to  distinguish  metacognition  from  reflection.  In  some  way 
mSSon  and  reflection  are  very  similar  since  both  of  them  involve  th.nk.ng  about  ones 
thinkinn  or  doina    Dewev  (1 933)  defined  reflective  thinking  as 

9«the  kind  of  thinking  that  consists  in  turning  a  subject  over  in  the  mind  and  giving  rt 
serious  and  consecutive  consideration.  It  involves  not  simply  a  sequence  of  ideas.but  a 
consequence  a  consecutive  ordering  In  such  a  way  that  each  determines  the  next  as  its 
proper  outcome,  while  each  outcome  in  turn  leans  back  on,  or  refers  to,  its 

As  deschbere\TerIhiPmoment.to.moment  action  required  in  managing  one's  learning  in  a 

*°™ slmi,arities  with  ref,ective  thinkin9-\ Howe  ,er;  \T 

ToSn^rVroaSlve  act  embodied  in  reflective  thinking,  metacognition  requires  the 
ISXXm  monitoring,  taking  the  form  ^r^^ 
mindfulness.   It  Is  reasonable  to  draw  the  conclusion  that  when  a  P^£^2"L  UDOn 
metacognitive  activity,  he/she  must  be  reflective.  However,  when  th.s  person  reflects  upon 
something,  he/she  may  not  necessarily  exercise  metacognition. 

Mfltacoanition  as  dynamic  and  spontaneous  mental  activity:    More  important, 

rather  nfflU- ■  °f  t£  Z2Xfi£^ 

in  a  larger  learning  context. 

Bfi~n^^  along  with  drifting  thoughts,  anc .keeping  in 

mind  how  a  novfce  surfer  becomes  adept  at  balancing  him/herself  in  the  unexpected  surges  (as 
Sed  ?hough^)  \  will  elaborate  in  the  following  section  as  to  how  one  becomes  a  person  from 
undfii  and  on  metacognition,  to  in,  and  then  pi  ^cognition. 

Although  most  of  us  use  metacognitive  thinking  in  some  form  or  other  we  are i  not 
necessarily  aware  of  it,  because  the  activity  is  in  our  subconsciousness.  This  .s  what  I  call 
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lindfil  metacognition.  When  we  are  under  metacognition,  drifting  thoughts  and  habitual  thinking 
and  acting  patterns  play  very  dominant  roles  in  driving  cur  action.  We  might  be  metacognltlve 
sometimes,  but  this  appearance  is  very  momentary.  Most  of  the  time,  our  learning  strategies 
are  used  habitually,  lacking  a  critical  reflection  of  their  virtues  and  vices.  Once  we  come  to 
recognize  the  existence  of  metacognition  and  begin  to  know  the  various  metacognltlve  skills  we 
start  the  Journey  to  the  practice  of  metacognition,  and  will  be  on  metacognition.  During  this 
stage,  strategies  play  a  more  explicit  role  than  they  do  in  the  previous  stage,  since  most  of  our 
attention  is  directed  toward  the  newly-known  strategies.  While  drifting  thoughts  and  habitual 
thought  patterns  are  still  active,  there  is  more  room  for  the  exercise  of  metacognition. 
However,  our  awareness  of  our  own  mental  states  and  traits,  or  use  of  the  newly  acquired 
strategies,  and  monitoring  process  are  frequently  interrupted  by  various  kinds  of  drifting 
thoughts.  When  we  enter  jnja.  metacognition.  we  are  first  confined  within  the  framework; 
metacognltlve  skills,  at  the  moment,  become  the  focus  of  our  attention.  However,  we  are  not 
able  to  use  them  freely  although  we  are  taught  to  or  learn  to  deliberately  apply  metacognltlve 
skills.  While  other  metacognitive  skills,  such  as  critical  awareness  and  reflection  are  also 
taking  a  more  active  form,  these  skills  are  more  external  than  internal.  It  is  not  until  we  can 
automatize  metacognitive  thinking  that  it  becomes  a  natural  part  of  our  thinking,  and  will  not 
impede  creativity.  Until  metacognitive  thinking  becomes  autonomous  and  used  less  deliberately 
then  we  will  be  gf  metacognition.  When  we  are  of  metacognition,  we  will  no  longer  stand  inside  ' 
the  circle  of  ourselves.  We  can  distance  ourselves  more  easily  and  look  into  ourselves  from  a 
wider  angle  and  more  diverse  perspectives.  Reflection  at  this  stage  will  become  more  critical 
then  it  is  in  the  previous  stages,  since  the  self  is  elevated  from  the  original  lower  point 
Although  drifting  thoughts  and  habitual  patterns  of  acting  and  thinking  might  occur  sometimes  at 
this  stage,  we  can  direct  these  interfering  thoughts,  confronting  them  with  more  positive 
attitudes  by  searching  for  the  rooted  problems  more  clearly,  and  convert  these  thoughts  into 
forces  of  improvement.  In  general,  the  process  is  not  linear,  but  cyclical.  Even  after  we 
become  of  metacognition,  we  might  be  undei  metacognition  for  short  moments  when  our  mental 
states  are  under  great  Impact. 

It  is  important  to  identify  where  a  learner  stands  In  this  process  and  provide 
appropriate  instruction  accordingly.  Because  metacognitive  development  takes  time  to 
cultivate,  neglecting  these  stages  will  result  in  futile  efforts  concerning  instruction  and  design 
It  is  also  essential  to  incorporate  instruction  and  design  of  instruction  with  such  acquisition 
processes,  giving  full  consideration  to  the  various  concepts  relevant  to  metacognition  In  other 
words,  the  design  of  metacognitive  instruction  should  be  integrative  with  the  learning  process 
and  be  compatible  with  other  thinking  and  acting  processes.  How  one  proceeds  from  novice  to 
expert  of  metacognition  and  what  roles  the  reconceptualized  components  of  metacognition  play  in 
this  process  are  the  important  considerations  teachers  and  designers  should  take  into  account  in 
order  to  make  instruction  rigorous. 

PRINCIPLES  OF  METACOGNITIVE  INSTRUCTIONAL  DESIGN 

Levels  of  empowerment 

Generally  speaking,  there  are  several  levels  of  empowerment.  For  example 
empowerment  can  take  place  at  the  personal,  Interpersonal,  organizational,  or  societal  level 
No  matter  wnien  level  it  might  occur,  it  will  definitely  Influence  other  levels  to  a  certain 
extent  Apparently,  the  developmental  process  of  metacognition  seems  to  belong  to  the  personal 
psychological  level.  However,  the  essence  of  empowerment  through  metacognition  is  to  make 
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changes  in  one's  thinking  and  ultimately  action.  When  people  at  each  level  become  more 
metacognitive,  empowerment  will  simultaneously  occur  at  the  various  levels,  which  in  turn 
makes  the  environment  more  favorable  to  the  fostering  of  metacognitive  development. 

Instructional  principles  ftf  flmpowarment 

Pluralistic  spirit:    The  first  principle  of  empowering  learners  through 
metacognition  is  that  instructional  and  design  must  have  a  pluralistic  spirit,  since 
metacognitive  thinking  differs  from  one  person  to  another.  However,  instructional  designers 
have  been  working  in  a  mode  of  totality,  certainty,  and  control.  Glroux  (1988)  points  out  that 
educational  theory  has  been  strongly  wedded  to  the  language  and  assumptions  of  modernism. 
Within  the  discourse  of  modernism,  knowledge  draws  Its  boundaries  almost  exclusively  from  a 
European  model  of  culture  and  civilization.  According  to  Lyotard  (1984),  postmodernism  is  a 
rejection  of  grand  narratives,  metaphysical  philosophies,  and  any  other  form  of  totalizing 
thought.  The  challenge  of  postmodernism  Is  Important  for  instructional  designers  because  it 
raises  crucial  questions  regarding  certain  hegemonic  aspects  of  modernism  and,  by  implication, 
how  these  have  affected  the  meaning  and  dynamics  of  our  practice.  In  recent  years,  more  and 
more  educational  technologists  have  recognized  that  the  standardization  mindset  of  the  industrial 
age  should  give  way  to  the  diversity  mindset  of  the  Information  age.  For  example,  Benathy 
(1991)  states  that  the  prevailing  educational  deslgn-whlch  Is  rooted  in  the  industrial  model  of 
the  last  century-cannot  be  Improved  or  restructured  to  meet  the  requirements  of  the  post- 
industrial  information/  knowledge  age.  We  are  called  upon  to  envisici  new  images  of  education 
and,  based  on  them,  design  new  systems  of  learning  and  human  development  which  will  prepare 
future  generations  for  the  challenges  of  the  new  era,  and  empower  them  to  shape  their  own  lives 
and  the  life  of  their  society.  It  is  my  belief  that  instructional  designers  should  recognize  the 
plurative  reality  in  the  postmodern  era  and  think  metacognitively  during  the  design  process  In 
order  to  broaden  or  even  break  the  boundaries  that  prevent  them  from  designing  creatively. 

Such  pluralistic  spirit  will  be  best  presented  if  instructional  designers  identify 
learners'  multiple  ways  of  knowing,  recognizing  the  various  human  beings'  intelligences 
without  favoring  only  some  of  them  (Gardner,  1985).  When  designing  instruction,  one  of  the 
Instructional  approaches  is  multiple  ways  of  teaching.  There  is  no  best  Instructional  strategy, 
but  only  most  appropriate  for  certain  types  of  learning  tasks,  ones  for  certain  group  of 
learners,  and  under  certain  circumstances.  Instructional  design  should  embrace  multiplicity, 
rather  than  optimality.  Taking  analogy  as  an  example,  a  pluralistic  way  of  designing  Instruction 
with  analogy  would  not  only  be  multiple  analogies  for  a  variety  of  contexts  (Wilson  &  Cole, 
1990),  but  also  analogies  that  can  accommodate  the  cultures  of  diverse  groups  of  learners, 
using  'the  language  and  social  norms  familiar  to  the  learners.  In  other  words,  analogies  should 
not  be  formulated  at  the  abstract  or  conceptual  level.  Instead,  it  should  be  integral  with  the 
target  learners'  culture. 

Resolving  the  boundary:  The  second  principle  is  resolving  the  boundary. 
Instruction  and  design  of  metacognition,  though  they  can  be  initiated  or  undertaken  from  the 
content  of  metacognition,  are  not  a  self-independent  activities.  The  first  step  for  us  to  resolve 
the  boundary  is  to  unlearn  what  we  have  learned  because  we  are  often  blinded  by  what  we  have 
known,  rather  than  what  we  don't  know.  The  second  is  to  approach  a  task  from  as  many  angles  as 
possible.  For  example,  metacognition  can  be  regarded  from  cognitive,  affective  or  behavioral 
aspects.  It  can  also  be  studied  whii  creativity,  critical  thinking,  problem  solving  etc.  In  the 
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section  of  reconceptualization  of  metacognition,  I  have  demonstrated  some  of  this  spirit. 
Limiting  the  design  or  instruction  of  metacognition  simply  to  the  content  of  metacognition  will 
result  in  rigidity  and  inflexibility.  This  is  to  see  a  task  ALWAYS  as  a  part  of  something. 

Divergent  inquiry:   In  order  to  empower  learners,  inquiry  should  take  divergent 
forms.  Any  point  could  be  the  best  cutting  point  for  instruction  and  design  as  long  as  it  leads  to 
the  goal.  Where  the  starting  point  of  inquiry  stK.ild  be  all  depends  on  the  learners1 
metacognitive  stage  and  level,  needs,  and  most  important  of  all,  teachers'  (designers) 
knowledge,  personality  and  capabilities.  It  is  important  that  instructors  or  designers  identify 
where  they  and  the  learners  are  standing  in  the  process  of  metacognitive  development,  and 
understand  the  most  difficult  metacognitive  skills  that  confront  learners  or  themselves  in  order 
to  determine  the  focus  of  instructional  emphasis.  It  is  also  essential  to  explore  the  dynamics  of 
their  and  learners'  development  in  the  process  since  metacognition  is  an  ongoing  mental 
activity.    Learners  should  be  exploring  this  collaboratively  with  instructors,  or  instructional 
designers  should  be  looking  into  this  with  instructors.  Because  learners  of  the  same  group 
might  be  in  various  stages  of  metacognition,  Instructional  guidance  and  support  should  vary 
accordingly.  Inquiry  approach  for  people  under  metacognition  could  be  more  direct  and 
concrete,  whereas  for  those  of  metacognition,  it  could  be  less  strategic. 

In  addition,  inquiry  can  be  an  Inside-out  or  outside-in  approach  or  other  kinds.  For 
example,  an  instructional  strategy  can  either  direct  learners'  attention  first  to  his/her  own 
drifting  thoughts,  examining  how  they  Influence  their  learning,  or  it  can  focus  on  learners'  task 
performance,  tracing  back  to  the  interfering  factors.  It  can  also  be  inquiry  of  others,  which 
illuminates  understanding  of  self.  Divergent  inquiry  strategies  could  ensure  both  the  breadth 
and  depth  of  multiplicity  and  reflectivity. 

Treasuring  mistakes:  The  essence  of  empowering  learners  through  metacognition 
lies  in  its  capability  of  turning  drifting,  disturbing  and  interfering  thoughts  to  positive,  guided, 
generating  and  even  creative  thoughts.  Therefore,  instruction  and  design  of  metacognition 
should  value  mistakes  and  misconceptions,  and  distorted  ideologies  for  they  are  the  sources  from 
which  possibilities  of  change  arise.  Ail  of  these  sources  should  be  highly  valued  and  be  traced 
back  so  that  rooted  problems  can  be  explored.  Teachers  or  instructional  designers  tend  to  value 
strengths  more  than  weaknesses  and  correctness  more  than  mistakes  in  the  process  of 
instruction.  Many  instructional  models  emphasize  how  to  make  the  best  use  of  learners' 
strengths  in  order  to  motivate  them  or  enhance  their  self-confidence  and  motivation.  However, 
there  is  no  instructional  approach  that  Is  a  cure-all.  In  many  cases,  people  need  to  be  aware  of 
the  internal  and  external  obstacles  that  inhibit  their  progress.  Instruction  and  design  of 
metacognition  should  make  these  explicit  to  learners. 

Strategy-reflection:    For  this  principle,  instructional  designers  or  teachers  are  not 
to  reflect  how  well  learners  have  performed  through  certain  strategies.  The  focus  is  not  the 
learners'  learning  achievement,  but  the  instructors  or  designers  themselves.  Instructional 
design  is  not  merely  the  design  of  a  learning  task.  It  is  how  designers  SEE  the  to-be-designed 
task  and  users/learners.  It  is  embedded  in  teachers'  (designers')  knowledge,  values, 
experiences  and  some  hidden  Ideologies  and  presumptions.  For  metacognition  is  a  more  abstract 
concept,  which  takes  one  form  or  another  in  everyone's  mind,  the  strategy-reflection  principle 
should  be  directed  to  reflect  the  values,  assumptions  and  ideologies  that  are  embedded  in  the  way 
we  teach  or  design.  By  doing  so,  we  can  be  emancipated  from  our  confined  limited  knowledge  of 
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learning,  instruction  and  design. 


CONCLUSION:  A  REMARK  ON  DESIGN 

In  this  paper,  I  have  proposed  a  new  perspective  for  designing  metacognition,  drawing 
both  upon  how  human  beings  acquire  metacognitive  skills  and  how  these  skills  are  interwoven 
with  other  thinking  skills.  It  helps  us  investigate  how  the  acquisition  of  metacognitive  skills 
could  be  coordinated  with  appropriate  approaches  to  teaching,  guiding  and  enlightening  learners 
about  these  same  skills.  An  instructional  designer  should  bear  the  dynamic  and  integrative 
nature  of  metacognition  in  mind,  so  that  the  actuation  of  learning,  instruction  and  design 
possibilities  can  be  attained.  In  other  words,  the  instruction  and  design  of  metacognition  should 
be  integrated  with  the  nature  and  development  of  one's  metacognition. 

Traditionally,  learning  theories,  instructional  theories  and  Instructional  design  theories 
are  treated  as  three  distinct  entities.  Numerous  arguments  about  whether  instructional 
theories  can  stand  alone  from  learning  theories  have  been  made.  Recent  exploration  of 
Constructivism  has  provided  a  great  deal  of  insights  into  the  possibilities  of  integrating  the  two. 
Nevertheless,  current  instructional  models  of  metacognitive  thinking  do  not  incorporate  tne 
learning  process  of  metacognition.  They  are  procedural-oriented,  neglecting  the  dynamic 
process  and  the  holistic  nature  of  mental  activities  because  of  the  problematic  definitions  and 
their  narrow  view  of  the  function  of  metacognition.  Since  metacognition  is  taught  without  being 
integrated  with  other  thinking  skills,  creativity  and  ease  of  learning  may  be  often  sacrificed. 

It  is  not  until  learners,  teachers  and  designers  all  engage  in  the  roles  they  play  that  a 
learning-centered  environment  becomes  possible.  Teachers  need  to  play  the  role  of  facilitator 
in  order  to  guide,  rather  than  control  students'  learning.  They  should  never  cease  to  be  learners 
if  they  wish  to  master  the  art  of  teaching.  Instructional  designers,  who  have  taken  for  granted 
that  their  knowledge  of  instruction  exceeds  that  of  teachers,  need  to  reflect  on  the  complex 
interactions  between  learning  and  instruction.  More  important,  learners  need  to  be  given  the 
responsibility  to  direct,  manage  or  even  design  their  own  learning.  When  they  become  teachers 
of  themselves,  they  could  be  more  creative.  In  addition,  teachers'  metacognition  will  enable 
them  to  perceive  their  instruction  in  a  more  reflective,  dynamic  way;  instructional  designers', 
together  with  learners'  metacognition,  will  allow  them  to  make  better  decisions  about  the  kinds 
of  instructional  and  learning  activities  appropriate  for  learners. 

The  mission  of  Instructional  designers  Is  to  enable  teachers  to  design  instruction 
metacognitively  that  will  in  turn  empower  students  to  design  learning  for  themselves.  Empower 
teachers  and  we  empower  learners.  In  a  learning-centered  environment,  instructional 
designers  will  be  the  backstage  heroes,  assisting  learners  and  teachers  to  become  the  main 
actors  of  the  learning  drama. 
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Introduction 


The  field  of  instructional  technology  is  in  transition.  During  the 
past  few  decades,  the  arrival  of  the  information  age  has  had  a  profound 
impact  on  the  field.  The  most  visible  change  has  been  the  dramatic 
development  and  extensive  availability  of  computers  and 
telecommunications  technologies.  The  concepts,  applications,  and 
resources  developed  from  these  technological  advances  when  applied  to  the 
field  greatly  stimulate  our  imagination  and  promote  our  capability  of 
solving  instructional  and  learning  problems. 

These  technological  advances  also  allow  us  to  think  beyond  public 
and  higher  education.  More  and  more  1ST  graduates  and  faculty  members 
are  taking  jobs  and  consulting  in  nontraditional  educational  contexts  such 
as  corporate  development,  government  training,  health  care  institutions, 
and  other  such  areas.  In  short,  the  field  of  instructional  technology  is 
dynamically  expanding  its  clientele.  While  there  arc  more  possibilities 
and  opportunities  being  generated,  at  the  same  time  there  are  more 
responsibilities  to  consider. 

In  order  to  run  our  business  more  effectively  and  wisely  in  the  new 
era,  we  need  to  have  a  new  perspective  which  can  furnish  new  guidelines. 
This  paper  is  thus  intended  to  synthesize  the  promising  instructional 
development  perspectives  into  a  current  set  which  might  be  discussed  by  the 
profession  at  large  and  might  highlight  new  directions  for  the  field. 

In  Search  of  a  New  Perspective: 

Da  vies  (1982)  argued  that  instructional  development  can  be  best 
understood  as  a  set  of  criteria  and  he  contended  that  each  process  of 
instructional  development  must  contribute  to  these  criteria.  The  final 
results  should  be  well  integrated  into  people's  lives  and  be  concerned  with 
not  only  their  behavior  but  also  their  cognitive  processes.  That  is, 
instructional  development  is  concerned  with  worthy  human  performance 
and  healthy  human  development. 

In  this  paper,  the  authors  propose  an  integrated  set  of  criteria  entitled 
"Configurational  Inquiry  Instructional  Development  Perspective  " 
(CIIDP).  By  the  term  "configuration;'  the  authors  mean  the  structure  and 
arrangement  of  a  certain  system  and  of  its  functional  processes.  The  word 
configuration  also  implies  that,  the  vigorous  interrelationship  and 
interaction  of  the  elements  existing  in  that  system.  The  term  "inquiry" 
usually  denotes  a  quest  or  investigation  for  knowledge  or  truth.  Nelson 
(1976)  interpreted  inquiry  as  a  means,  whether  scientific  or  philosophical 
utilized  by  investigators  to  be  fruitful;  that  is  it  is  used  to  solve  inquirers'' 
problems  through  a  thorough  investigation  toward  the  research  target.  In 
snort,  "configurational  inquiry"  means  that  a  thorough  detection, 
diagnosis,  and  appraisal  of  a  variety  of  huiuun  or  non-human  systems  in 
terms  of  the  structure,  arrangement  and  interrelationship  so  as  to  resolve 
the  inquirer's  problems. 

The  CIIDP  encomprahes  four  features  which  include  (1)  systemic 
thinking  as  its  structure,  (2)  interdisciplinary  approach  as  its  path,  (3) 
human-based  inquiry  as  its  heart,  and  (4)  improving  interfaces  as  its  top 
task. 

First  or  all,  instructional  development  is  not  a  concept  like 
mechanics  -  it  is  more  like  liberty  fDavies,  1982.)  The  exploration  of  a 
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complex  concept  such  as  liberty  thus  requires  the  use  of  a  more  sophisticated 
method  such  as  the  holistic  nature  of  the  systemic  approach.  Therefore,  one 
of  the  determinants  for  an  instructional  developer  to  succeed,  the  authors 
assert,  is  to  undertake  a  systemic  inquiry  of  the  configuration  of  various  ID 
processes. 

Second,  instructional  development  could  be  regarded  as  an  activity 
to  facilitate  human  development  through  the  improvement  of  instructional 
and  learning  process.  However,  human  development  is  so  complex  that 
adequate  understanding  cannot  result  only  from  one  specialization.  In 
other  words,  the  need  of  an  interdisciplinary  approach  for  instructional 
development  is  needed  in  the  areas  of  theory  and  practice.  One  of  the 
purposes  of  this  paper  is  to  remind  and  encourage  practitioners  of 
instructional  technology  to  rethink  the  roots  of  our  field  and  to  enrich  these 
roots  through  the  inquiry  of  the  configuration  of  polydisciplines. 

Third,  instructional  developers  are  social  activists  in  scientists 
clothing  who  mainly  perform  their  jobs  among  people  (Schwen  et  al.,  1984). 
The  jobs  are  basically  a  task  of  human-based  inquiry  which  involves  a 
great  deal  of  interaction,  communication,  and  negotiation  within  and 
among  the  various  human  communities.  The  authors  argue  that 
instructional  development  should  be  neither  instructor-  nor  learner 
centered,  but  rather  human  centered.  By  human-centered,  the  authors 
mean  that  all  significant  members  of  the  ID  process  are  taken  into  account. 
An  appropriate  exploration  of  the  configuration  of  a  given  human  society 
thus  becomes  another  significant  element  in  the  ID  process. 

Finally,  in  order  to  optimize  the  development  of  instruction,  we  need 
to  polish  the  interfaces  among  all  identities  involved  in  the  process.  These 
interfaces  are  composed  of  various  gaps  and  obstacles.  In  order  to  resolve 
instructional/learning  problems,  we  need  to  overcome  these  barriers,  so  as 
to  improve  these  interfaces.  In  other  words,  we  need  an  inquiry  of  the 
configuration  of  diverse  interfaces  toward  the  optimal  organization  of 
instructional  systems. 

Systemic  Thinking  as  Its  Structure 

Davies  (1991)  compared  two  product  planning  strategies,  parallel 
development  vs.  serial  development  (see  figure  1),  and  explored  why 
instructional  developers  need  to  adopt  the  parallel  development  method.  He 
illustrated  that  adopting  a  parallel  development  approach  enables  us  to 
reduce  the  cycle  time,  remove  the  number  of  interfaces  between  the  ID 
functions,  and  conceptualize  the  ID  process  as  an  art  rather  than  an 
engineering  activity.  In  essence,  the  parallel  development  method  is  a 
systemic  approach,  and  the  serial  one  is  systematic. 


Insert  Figure  1  Here 


In  addition,  the  authors  contend  that  the  followings  can  complement 
Davies's  notion  and  make  it  more  complete.  Since  a  systemic  approach  is 
based  on  social  logic  and  adopts  the  limited  rational  philosophy,  it  thus  is 
closer  to  the  social  reality.  On  the  other  hand,  a  systematic  approach  is 
based  upon  scientific  logic  and  adopts  an  absolutely  national  concept,  so  it  is 
suitable  for  highly  controlled  experimental  situations.  While  a  systemic 
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approach  optimizes  feedback  and  accelerates  revision,  a  systematic 
sacrifices  feedback  and  slows  down  revision.  A  systemic  approach  places 
greater  emphasis  on  the  dynamics  of  participation;  however,  a  systematic 
approach  lacks  a  cooperative  climate  among  the  participants.  A  systemic 
approach  gives  us  more  choices  which  makes  the  process  more 
personalized;  in  contrast,  a  systematic  approach  formulates  a  set  of  rigid 
procedures  which  inhibits  our  creativity  and  imagination. 

The  discussion  of  systematic  and  systemic  methods  among 
instructional  technology  professionals  is  not  new.  While  most  of  them 
focus  upon  the  differences  and  application  between  these  two  perspectives, 
there  is  little  in  the  literature  that  examines  their  relationship.  In  order  to 
synthesize  and  make  a  connection  among  these  contributions,  the  authors 
argue  that  the  systematic  approach,  in  fact,  can  be  regarded  as  a  subset  of  the 
systemic  approach.  By  adopting  this  concept,  we  can  formulate  a  particular 
strategy  according  to  a  specific  time  at  a  specific  place.  We  hold  highly 
tuned  analytical  knowledge  and  skills  which  make  our  performance 
artful.  We  thus  can  make  a  wiser  and  more  realistic  decision  in  light  of 
the  situation  encountered. 

For  instance,  when  the  circumstance  is  less  freedom  or  the  scenario 
is  a  pedagogic  one,  especially  when  there  are  low-level  skills  to  be  taught, 
we  can  accordingly  construct  a  highly  structured  procedure  to  develop 
instruction  because  such  a  systematic  approach  gives  people  basic  skills 
and  it  is  easy  to  follow.  Devising  a  military  training  program  is  a  case. 
On  the  other  hand,  we  can  also  conduct  an  ID  process  like  rapid  prototyping 
(Tripp  et  al.,  1990)  to  meet  a  certain  clients  request.  Such  an  alternative 
strategy  is  especially  suited  to  a  decentralized  clieni  system  in  which 
instructional  developers  and  clients  work  together  in  a  participative, 
interactive  mode.  To  work  with  a  systemic  approach,  we  rearrange,  or 
sometimes  remove,  and  prioritize  instructional  development  phases 
accordingly  to  optimize  our  efforts;  that  is,  we  work  in  a  flexible  manner. 

Instructional  development  is  a  dynamic  activity  which  involves  a 
myriad  of  decisions  and  judgments.  Adopting  a  systemic  approach  enables 
us  to  quickly  penetrate  the  problems  and  make  accurate  judgments  and 
wise  decisions.  This  adoption  is  especially  meaningful  in  the  information 
age  which  is  both  content  and  context  rich.  In  addition,  a  systemic  approach 
facilitates  implementation,  liberates  the  structure,  pays  attention  to  the 
relationship  and  interaction  of  all  involving  participants,  and  focuses  upon 
the  matters  of  elegance,  effectiveness  and  efficiency.  All  of  these  meet  the 
contemporary  demands  of  an  instructional  development  activity  in  terms 
of  both  theory  and  practice. 

Bholas  (1991)  notion  regarding  this  issue  is  particularly  relevant: 
"the  systematic  is  merely  a  scaffolding  for  developing  a  socio-logic;  a 
systemic  of  purposive  action  that,  in  the  real  world  of  practice,  can 
accommodate,  the  emotion,  the  intuitive,  the  reflective,  the  social  and  the 
structural."  The  advocacy  of  a  systemic  inquiry  of  the  configuration  of 
instructional  development  process  now  seems  both  appropriate  and  timely. 

Interdisciplinary  Approach  as  Its  Path 

The  nature  of  instructional  technology  is  eclectic.  People  in  the 
field  come  from  diverse  disciplines  and  have  varied  expertise  to  offer. 
Although  the  concept  of  the  interdisciplinary  approach  is  not  new  to  our 
field,  it  has  been  largely  overlooked.  When  performing  an  instructional 
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development  task,  we  usually  stick  to  our  own  paradigms  and  use  our  own 
expertise  in  problem  solving  with  little  objective  observation  and 
unprejudiced  analysis.  We  have  failed  to  think  globally  and  to  make 
connections  among  our  multifarious  and  profound  competencies. 

In  the  field  of  instructional  technology,  specialization  is  still 
needed  at  certain  levels,  but  innovative  solutions  cannot  be  created  by 
merely  one  specialization.  That  is,  by  looking  at  multiple  perspectives,  we 
can  secure  the  benefits  which  are  generated  from  shared  and  wise 
solutions.  The  cooperative  interaction  between  diverse  disciplines  can 
complement  and  extend  the  development  of  new  knowledge  and  its 
application  to  the  field.  The  interdisciplinary  approach  can  help  us 
investigate  known  systems  in  a  creative  search  for  new  systems  which  will 
invent  new  problems;  that  is,  this  approach  allows  us  to  go  beyond  problem 
solving  to  a  level  of  problem  finding. 

Working  in  an  interdisciplinary  approach  is  to  perform  our  jobs 
within  a  semiotic  framework  which  can  be  illustrated  as:  thesis  -> 
anything  else  that  seems  appropriate  (including  anti-thesis)  ->  synthesis. 
For  example,  what  can  instruction*!  developers  learn  from  a  professional 
landscape  photographer?  What  competencies  of  a  basketball  player  are 
worthwhile  for  us  to  master?  What  characteristics  of  a  marketing  manager 
can  be  of  benefit  to  us?  A  professional  landscape  photographer's  love  of 
his/her  subject  and  the  ability  to  simplify  the  complexity  and  to  anticipate 
the  optimal  arrangement;  a  basketball  player's  superb  skills,  flexibility, 
energy,  and  a  keen  reflex;  and  a  marketing  manager's  ability  to  recognize 
the  maximum  entry  point  to  the  identified  target,  and  a  sensibility  to  the 
sophisticated  reality  and  a  holistic  view-all  of  these,  can  be  of  benefit  to  us 
when  performing  our  jobs.  By  doing  this,  we  can  maximize  our  strengths, 
experiences,  and  resources.  The  combined  efforts  may  also  contribute 
added  dimensions  to  our  research  and  applications.  The  convergence  of 
independent  ideas  is  viewed  as  being  more  substantial  than  ideas  would 
have  been  singularly. 

In  addition  to  providing  the  combined  efforts,  an  interdisciplinary 
approach  as  tiie  path  of  instructional  development  has  another  implication. 
It  encourages  us  to  think  outside  of  "the  box";  that  is,  it  urges  us  to  be 
sensitive  to  the  surrounding  environment  and  to  think  globally.  Schwen 
(1988)  indicated  that  the  most  significant  determinants  of  our  professional 
futures  are  outside  of  our  control  in  many  ways.  Outside  our  control  does 
not  mean  that  we  should  disregard  these  factors,  instead  it  implies  that  we 
must  be  quite  sensitive  to  these  social,  physical,  and  intellectual  contexts 
and  forces.  Being  sensitive  to  the  environmental  reality  means  that  we 
should  view  reality  as  the  complete  interdependence  and  nonseparability  of 
variables.  In  other  words,  we  should  treat  these  uncertainties  as  a 
characteristic  of  problems  and  resolutions.  Being  sensitive  to  the 
environmental  reality  also  means  that  we  should  become  more  open  to 
alternatives  and  integrate  contradictions  into  an  all-inclusive  whole 
which  is  a  basis  of  reality.  A  wise  application  of  the  environmental 
influences  would  empower  us  in  terms  of  elegance,  effectiveness,  and 
efficiency. 

Studies  on  humans  and  their  societies  necessarily  transcend 
discipline  boundaries.  Batts  (1986)  indicated  that  whenever  knowledge  is  to 
be  used  interpretively  or  applicatively,  curriculum  content  will  tend  to  be 
determined  by  real-life  situations,  and  the  network  of  ideas,  concepts, 
patterns,  or  relationships  pursued  will  be  largely  determined 
pragmatically.  While  the  practice  of  an  interdisciplinary  study  facilitates 
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knowledge  transfer,  an  application  of  interdisciplinary  approach  enables 
instructional  developers  to  proact  and  react  in  the  reality.  A  thorough 
inquiry  through  an  interdisciplinary  approach  helps  us  anticipate  potential 
obstacles  (for  elegance),  reduce  the  cycle  time  (for  efficiency),  and  identify 
more  opportunities  (for  effectiveness). 

Human-Based  Inquiry  as  Its  Heart 

In  order  to  optimize  the  instructional  development  process,  a 
human-based  inquiry  is  needed  in  terms  of  theory,  practice,  and  our  role. 
Education  is  not  about  something  casual,  but  about  the  proper  way  to  live. 
Schools  are  not  places  for  preparing  children  for  life,  Davies  (1976)  argued, 
they  are  life  themselves  for  children,  teachers,  and  parents.  He  went  on  to 
claim  that  we  need  a  people-oriented  school,  rather  than  a  child-centered  or 
teacher-centered  one.  Davies  (1982)  further  argued  for  a  shift  of 
instructional  development  movement.  He  claimed  that  the  new  generation 
of  instructional  development  is  both  organic  and  human  in  its  orientation. 
It  is  concerned  more  with  values,  situations,  choices,  and  roles  which 
mainly  result  from  human  inner  thoughts  and  outward  responses.  It 
respects  diverse  individual's  experience,  background,  and  culture.  It  also 
implies  a  need  of  human-based  inquiry  to  create  a  human-centered 
learning  environment. 

When  examining  our  role,  Schwen  et  al.  (1984)  indicated  that 
instructional  developers  are  social  activists  in  scientists'  clothing.  The 
implication  of  this  is  twofold.  First,  clothing  reflects  the  image  which  we 
reveal.  Clothing  also  signifies  the  methods,  tools  and  strategies  which  we 
carry  and  employ.  We  dress  up  and  apply  the  scientific  methodologies  and 
tools  to  address  the  human  learning  and  performance  problems.  Second,  it 
indicates  the  essence  of  our  role;  that  is,  we  are  social  activists  who  mainly 
perform  our  job  among  people.  The  job  is  basically  a  task  of  human-based 
inquiry  which  involves  a  great  deal  of  communication,  interaction,  and 
negotiation  within  various  human  communities. 

Banathy  (1991)  supported  this  standing  point  by  claiming  that  we 
have  seen  the  emergence  of  four  generations  of  design  approaches  to  human 
activity  systems.  While  the  first  generation  of  design  is  a  "design  by" 
approach,  the  second  "design  for",  and  the  third  "design  with",  the  fourth 
generation  is  an  approach  of  "designing  within."  Banathy  contended  that 
to  be  authentic  and  sustainable,  human  activity  systems  must  be  designed 
and  accomplished  by  the  creative  participation  of  all  people  in  the  system; 
that  is,  it  should  reflect  their  collective  vision  of  humanity.  Such 
participation,  Banathy  went  on  to  argue,  enables  people  to  better  understand 
their  system  and  their  role  in  it.  It  also  generates  consensus  among  those 
who  participate,  and  further  ensures  that  people  will  take  part  more 
effectively  and  with  a  deeper  level  of  commitment  in  the  implementation  of 
the  design.  He  encourages  every  community  member  by  stating  that 
designing  our  future  is  our  responsibility  and  we  should  take  charge  of 
Bhaping  it,  This  responsibility  should  thus  fall  on  each  of  us:  instructional 
developers,  learners,  instructors,  and  other  community  members.  This 
understanding  enables  self-determination  and  respects  the  autonomy  of  the 
community.  It  also  bestows  on  the  community  a  unique  self-guided  and 
self-directed  life.  In  short,  the  "design  within"  approach  gives  the 
community  ownership. 
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In  order  to  bestow  ownership,  we  should  conduct  instructional 
development  activity  within  the  client  system  and  involve  all  people  who 
are  in  it,  who  use  it,  and  who  are  served  by  it.  Therefore,  we  need  to  have  a 
human-oriented  instructional  development  model  which  is  grounded  by  a 
thorough  human-based  configuration  inquiry;  that  is,  it  should  be  central  to 
the  innate  potentialities  of  human  beings.  Theoretically,  the  heart  of  this 
model  should  be  grouped  with  a  set  of  people  including  the  learners,  the 
instructors,  the  content  experts,  the  instructional  developers,  and  other 
contemporary  life  such  as  the  school  administrators,  corporate  personnel  in 
the  community,  and  learner's  family,  friends,  etc. 

A  human-based  inquiry  can  enrich  ID  process  in  many  ways.  For 
instance  the  curriculum  revision  project  conducted  by  Indiana  University, 
Instructional  Systems  Technology  Department  (19894992),  provides  a  good 
example  in  applying  contemporary  life  to  identify  reality.  During  the 
initial  needs  assessment,  a  pool  of  IU  1ST  graduates  and  contemporary 
practitioners  in  the  field  actively  assisted  in  identifying  the  "real"  needs 
such  as  more  emphasis  upon  communications  skills,  better  preparation  in 
understanding  the  "bottom  line"  of  the  business  environment,  and  more 
exposure  to  and  "hands-on"  experience  with  the  emerging  technologies. 
The  IU  1ST  Multi-Media  labs  sponsored  by  the  Apple  and  AT&T 
Corporations  also  demonstrates  how  contemporary  life  can  play  a  worthy 
role  on  information  provider  and  resource  linker. 

In  addition,  the  establishment  of  the  NASDC  (The  New  American 
Schools  Development  Corporation)  illustrates  the  benefit  of  alliance  of 
business  and  education  (Reigeluth,  1991).  The  NASDC  was  formed  by 
business  leaders  with  the  hope  of  reinventing  American  education  by 
designing  new  schools  for  a  new  century.  It  is  an  independent,  non-profit 
organization  that  is  expected  to  raise  roughly  $200  million  from  the  private 
sector  to  underwrite  the  design  of  a  new  high-performance  educational 
environment. 

Moreover,  the  effort  and  achievement  of  "Education  2000  in  the 
United  Kingdom  give  us  another  persuasive  instance  to  incorporate  a 
human-based  inquiry  into  the  practice  of  instructional  development. 
Education  2000  was  formed  in  1982  by  a  group  of  individuals  from 
education,  industry,  and  commerce.  Its  aim  is  to  reshape  education  to  meet 
the  present  and  future  needs  of  individuals  within  the  society.  At  present, 
there  are  nine  Education  2000  learning  communities  established  in  the  UK. 
These  communities  all  have  around  65,000  residents  and  the  project 
involves  all  of  the  schools  and  all  community  members.  Strong 
community  involvement  (as  the  stimulus  for  change),  curriculum  revision 
(as  the  means  for  the  change),  and  technology  innovations  (used  to  support 
the  change)  are  its  basic  approach. 

As  indicated  above,  the  process  of  instructional  development  is  the 
collective  challenge  of  people  in  the  community.  If  we  want  our 
contribution  to  be  significant,  productive,  and  useful  to  our  society, 
intelligent  strategies  should  be  devised.  The  strategies  suited  to  this 
perspective  include  dynamic  participation,  flexible  grouping,  team 
collaboration,  experience  connecting,  resource  linking,  discipline 
integration,  creativity  development,  and  responsibility  taking. 
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Improving  Interfaces  as  Its  Top  Task 

While  the  term  "interface"  generally  means  the  space  where 
humans  and  machine  meet,  the  authors  interpret  it  as  an  area  in  which 
diverse  systems  contact,  communicate  and  interact  In  addition  to  human- 
machine  interfaces,  there  are  also  a  variety  of  interfaces  existing  between 
different  machines  or  difforent  human  groups.  For  our  purpose,  this  paper 
will  mainly  address  the  latter  which  varies  in  names  such  as  perception, 
experience,  belief,  value,  norm,  and  culture.  . 

Our  society  is  undergoing  massive  changes  as  we  become  more 
effected  by  the  information  age.  These  changes,  in  such  an  environment, 
can  be  viewed  as  having  a  rearrangement  of  roles,  relationships, 
responsibilities,  functions,  opportunities,  and  boundaries  within  or  among 
various  human  societies.  We  modify  or  reform  the  old  system  in  order  to 
adopt  changes,  but,  on  the  other  hand,  we  need  to  maintain  a  certain  amount 
of  harmony  as  the  changes  may  result  in  conflicts.  To  keep  harmony  is  to 
make  contacted  interfaces  compatible. 

These  interfaces  are  always  composed  of  reasonable  amount  of,  if 
not  many,  gaps  and  barriers.  In  order  to  adopt  changes  to  resolve  problems 
and  maintain  the  needed  harmony  simultaneously,  we  must  identify, 
prioritize,  and  overcome  these  obstacles;  that  is,  we  need  to  improve  these 
interfaces  in  terms  of  mayor  functions  and  major  roles.  The  authors  hold 
that  the  greatest  success  in  our  field  will  result  from  improving  these 
interfaces  rather  than  from  a  dramatic  breakthrough  in  hardware.  After 
all,  technology  must  be  accessible  to  be  useful.  Improved  interfaces  can 
bridge  a  variety  of  human  or  non-human  obstacles  in  order  to  maximize  the 
access  and  potentialities  of  technology.  The  success  of  instructional 
development  depends  upon  a  synthesis  effort  to  perfect  the  interfaces  among 
those  factors  involved  in  the  entire  process.  In  other  words,  we  need  a 
thorough  inquiry  of  the  configuration  of  diverse  interfaces  toward  the 
optimal  organization  of  instructional  systems. 

Arnn  and  Strickland  (1975)  remarked  that  frequently  strategies 
which  seem  excellent  on  paper  fail  to  produce  the  desired  results,  because 
they  lack  the  consideration  of  human  obstacles.  The  possible  obstacles 
according  to  Gentry  and  Trimby  (1984),  include: 

-  Gatekeeper  rejection  after  an  ID  project  is  underway 

-  Client  misinterpretation  of  ID  project  goals  and  activities 

-  Incongruities  within  a  client  organization 

-  Loss  of  momentum  after  initiation  of  a  project 

-  Conflicting  expectations  within  the  client  group 

-  Raid8  on  resources  previously  committed  to  an  ID  project 

-  Unclear  Client^Developer  lines  of  responsibility  and  authority 

-  Difficulty  in  obtaining  essential  information 

A  task  of  interface-improving  must  be  performed  throughout  the 
whole  activity  of  instructional  development.  The  authors  thus  view 
interface-improving  as  a  continual  psychological  and  physiological 
process.  The  strategies  associated  with  this  process  include  obstacle 
removing,  obstacle  rearranging,  and  obstacle  transforming  or  a 
combination  according  to  the  nature  of  the  obstacles. 

Obstacle  Removing:  Whenever  encountering  obstacles,  the  first 
thing  that  usually  comes  to  mind  is  to  remove  it  or  to  handicap  its  influence, 
strategies  such  as  promoting  communication,  redesigning  facilities 
supplying  adequate  materials  and  tools,  outfitting  appropriate  and  timely 
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information  or  feedback,  making  the  ID  process  more  interesting  and 
rewarding,  or  providing  proper  training  or  education,  can  remove,  or  at 
least  reduce,  a  certain  amount  of  obstacles. 

Obstacle  Rearranging:  For  those  obstacles  which  cannot  be  easily 
removed  or  need  to  be  kept  for  some  reason,  we  may  want  to  rearrange  them 
in  order  to  decrease  their  impact.  It  is  worthwhile  to  remember  thatan 
organization  may  derive  a  dynamic  spirit  from  the  existence  of  certain 
obstacles.  A  delicate  rearrangement  and  management  of  these  obstacles 
will  positively  facilitate  our  performance.  Flexible  teambuilding 
according  to  the  tasks,  more  appropriate  personnel  selection  and 
assignment  giving,  or  work  flow  and  load  redistribution,  are  tactics 
identified  in  this  approach.  . 

Obstacle  Transforming:  Many  unsatisfactory  performances  result 
not  from  human  factors,  although  sometimes  they  do,  but  rather  the 
organizational  structure  does  not  function  as  we  expect .  These  obstacles 
which  handicap  human  performance  exist  in  the  contexts  such  as  an 
organization's  personnel  structure,  operant  management,  information 
flow  and  identified  culture  which  cannot  be  simply  removed  or 
rearranged.  They  need  to  be  transformed.  In  order  to  transform  these 
obstacles,  we  need  to  reform  the  personnel  structure,  change  the 
management  practice,  redesign  the  information  flow,  or  alter 
organizational  culture,  norms,  and  beliefs.  Breaking  customary  thoughts 
and  practices  is  the  key  to  transform  this  kind  of  obstacles. 

A  five-year  transition  plan  to  whole  language  in  grades  K-8 
proceeding  in  the  Westwood  school  district,  Massachusetts,  provides  a 
concrete  example  for  this  situation  (Gursky,  1991).  Being  aware  that  a 
piecemeal  change  cannot  result  in  true  improvement,  Superintendent 
Robert  Monson,  seconded  by  a  teacher-led  reading  committee^  restructured 
the  school  system  which  move  away  from  basals  and  standardized  tests 
toward  literature  reading  and  alternative  forms  of  assessment  In 
addition,  the  restructuring  effort  of  Skowhegan  Area  Middle  School,  Maine, 
is  another  instance  (Norris  and  Reigeluth,  1991).  Under  the  plan  of 
schools-within-a-school,  innovative  instructional  activities  are  being 
implemented  out  to  meet  students'  needs  such  as  a  more  flexible  schedule, 
multi-grade  grouping,  and  so  forth.  Transforming  the  traditional 
schooling  obstacles  and  providing  a  better  interface  for  students  to  meet 
their  individual's  needs  greatly  facilitate  leaning. 

Improving  interfaces  is  an  endless  job.  It  also  is  an  exploration 
into  the  nature  of  both  human  beings  and  the  world  we  are  li  ve  in.  Many 
decisions  made  in  the  pursuit  should  be  aesthetic,  experiential,  and 
phenomenological.  This  pursuit  also  reveals  the  beauty  of  instructional 
development  as  an  art.  The  understanding  and  application  of  interface 
improving  as  a  vehicle  to  facilitate  human  development  and  improve 
human  performance  is  a  process  of  continual  refinement.  Not  only  do  we 
wish  to  make  contributions,  but  also  we  want  to  make  them  effective  and 
efficient,  elegant  and  graceful. 

Synthesis 

The  Configurational  Inquiry  Instructional  Development 
Perspective  stresses  that  we  need  a  thorough  inquiry  into  the  configurations 
of  polydisciplines,  various  ID  processes,  human  societies,  and  diverse 
interfaces  in  order  to  achieve  an  optimal  organization  of  instructional 
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systems.  The  inquiries  into  manifold  configurations  further  implies  that 
instructional  development  should  be  an  activity  of  process  emphasizing, 
goal  focusing,  dynamic  participation,  interface  improvement,  resource 
linking,  discipline  integration,  responsibility  taking,  and  glory  sharing 

Banathy  (1987)  indicated  that  an  instructional  development 
activity,  in  its  practice,  is  a  process  of  feedback-feedforward  and 
divergence-convergence.  In  order  to  shape  development  images,  we 
oftentimes  go  back  to  previous  formulations  and  re-explore  the  impact  of  the 
emerging  resolutions  upon  the  identified  problems,  but,  on  the  other  hand 
in  an  anticipatory  mode,  we  move  forward  and  speculate  about  the  effect  of 
current  design  and  choices  which  are  still  ahead  of  us.  In  addition,  we  call 
for  exploring  more  alternatives  in  the  beginning,  and,  as  the  activity 
proceeds,  continuing  to  search  for  opportunities  to  make  changes  as  we 
gradually  converge  toward  the  resolutions.  The  dynamics  of  divergence- 
convergence  are  revealed  while  the  developers  continually  go  through 
alternating  sequences  of  generating  variety  and  reducing  variety,  while 
seeking  the  most  feasible  alternative. 

As  figure  (2)  illustrates,  an  instructional  development  activity 
should  be  quite  sensitive  to  the  environmental  influence  such  as  the  social, 
physical,  and  intellectual  forces  and  contexts  surrounding  the  identified 
systems.  These  external  powers  provide  diverse  realities  and 
uncertainties  to  facilitate  or  obstruct  the  instructional  development  process. 
Both  learning  objectives  and  human  or  non-human  obstacles  are 
milestones.  The  instructional  development  components  are  the  journey. 
These  process  components  such  as  management,  analysis,  design, 
development,  implementation,  evaluation,  and  revision  are  carried  out 
through  iterative  cycles  as  we  examine  and  re-examine  the  centered 
human-base  and  their  goals.  When  we  reach  each  specific  objective  or 
overcome  a  particular  obstacle,  the  milestone  becomes  part  of  the  journey 
and  we  can  then  move  beyond  that.  These  recurring  spirals  are 
continually  ongoing  which  bring  forth  progressively  more  refined 
characterizations  of  the  system  to  be  developed.  This  also  is  a  process  of 
negotiation  among  involved  identities  with  different  viewpoints  and 
values  in  order  to  work  out  a  mutual  resolution.  As  Banathy  identified, 
dia  ectics  is  the  underlying  philosophy  of  this  approach,  and  consensus- 
building  methods  provide  the  technology.  Through  this  participatory 
process,  integrated  vision  could  be  established,  interfaces  could  be 
improved,  and,  the  most  important,  we  would  arrive  at  wiser  decisions 


Insert  Figure  2  Here 


While  we  accept  cultural  pluralism  as  a  viable  societal 
arrangement  and  experience  diversification  of  life-styles,  this  perspective 
provides  us  an  organic  framework  which  places  a  great  deal  of  emphasis 
upon  the  full  development  of  human  potential  and  their  cooperative 
mteraction.  This  perspective  is  both  synthesis-  and  decision-oriented. 
This  is  a  dynamic  model.  This  is  also  a  model  of  growth 
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Conclusion 

Progressing  from  "The  Age  of  Uncertainty"  (1977),  what 
inspiration  does  Galbraith's  philosophical  transcendence  of  The  Age  of 
Pragmatism"  (1992)  bring  to  us?  Pragmatism  implies  that  we  work  without 
a  restricted  paradigm.  It  also  implies  that  we  are  flexible  when  we  work 
with  a  given  reality.  An  attitude  of  respect  toward  societal  pluralism  is 
needed  to  make  decisions  in  terms  of  maximum  usefulness  in  order  to 
achieve  our  goals  and  satisfy  our  needs. 

There  is  no  single  best  way  to  develop  instruction,  Davies  (1984) 
claimed,  appropriateness  is  the  key  issue.  Working  without  a  restricted 
paradigm  is  the  first  step  in  identifying  appropriateness.  Appropriateness 
is  not  determined  by  the  developers,  instructors,  or  learners,  but  rather  by 
the  the  whole  human  community.  They  interpret  our  messages  in  the 
context  of  their  own  experiences  and  knowledge,  and  construct  meaning 
relative  to  their  own  needs,  background,  and  interests  (Jonassen,  1991). 

Pragmatism  also  signifies  an  attitude  of  "complex  in  mind,  but 
simple  on  hand"  as  we  communicate  with  our  clients  and  learners, 
especially  during  the  initial  stage.  On  the  one  hand,; we  regard  the 
complicated  realities  and  capricious  environmental  factors  as  our 
opportunities  and  integrate  diverse  contradictions  into  an  all-inclusive 
whole.  On  the  other  hand,  we  might  want  our  language  and  tools  to  be  as 
simple  and  compatible  as  possible  since,  by  doing  this  way,  a  mutual 
trustworthiness  between  the  innovation  and  adoption  systems  could  be 
easily  established  and  a  sense  of  accomplishment  would  also  be  developed. 
Thi3  attitude  further  suggests  that  we  should  be  aware  of  our  benefits  and 
limitations.  By  knowing  our  strengths  and  weaknesses,  consequently,  we 
would  arrive  at  more  precise  decisions. 

The  elements  of  the  Configurational  Inquiry  Instructional 
Development  Perspective  are  not  new,  but  the  synthesis  of  these  elements 
does  provide  a  new  orientation  and  offer  an  appealing  focus.  Due  to  its 
inner  complexity,  the  authors  suggest  that  a  pragmatic  attitude  should  be 
held  while  adopting  this  perspective.  In  addition,  the  authors  also  believe 
that  the  significance  of  this  perspective  meets  the  contemporary  needs  which 
illuminates  not  only  what  the  instructional  development  should  be,  but  also 
how  inquiry  into  the  instructional  technology  field  should  be  undertaken. 

In  closing,  the  authors  would  assert  that  while  our  field  is  in 
transition,  we  all  have  real  opportunities  and  responsibilities  which  are 
essential  to  our  growth  and  development.  The  Configurational  Inquiry 
Instructional  Development  Perspective  functions  as  a  catalyst  to  facilitate 
this  growth  and  development. 
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Teaching  Self-Regulated  Taawiing  Strategies 

Learning  is  a  complex  process,  one  which  many  students,  despite  years  of 
schooling,  still  find  mysterious  (Thomas  &  Rohwer,  1986).  What  distinguishes  the 
successful  student  from  his/her  less  successful  peers?  A  growing  body  of  literature 
supports  the  notion  that  optimal  academic  performance  is  strongly  tied  to  the  degree  of  self- 
regulation  the  learner  is  capable  of  exercising  (Borkowski,  et.  al.,  1990;  Jones  &  Idol,  1990; 
Zimmerman  &  Pons,  1986;  Zimmerman,  1990).  Although  the  self-regulated  learning 
perspective  is  not,  from  a  theoretical  position,  a  unified  one,  according  to  Zimmerman 
(1990,  p.4),  "a  common  conceptualization  of  these  students  has  emerged  as 
metacognitively,  motivationally  and  behaviorally  active  participants  in  their  own 
learning."  In  other  words,  self-regulated  learners  are  purposive  and  goal  oriented 
(proactive  rather  than  simply  reactive),  incorporating  and  applying  a  variety  of  strategic 
behaviors  designed  to  optimize  their  academic  performance. 

While  many  students,  barring  those  who  are  totally  tuned  out,  are,  to  varying 
degrees,  active  in  the  manner  just  described,  self-regulated  learners  appear  to  be  both  more 
keenly  aware  of  the  relation  between  specific  behaviors  and  academic  success  and  more 
likely  to  systematically  and  appropriately  employ  such  behaviors  (Zimmerman  &  Pons, 
1986).  They  also  exhibit  greater  flexibility  in  adapting  to  the  variable  and  sometimes 
uncertain  demands  that  exist  in  the  classroom,  particularly  at  the  high  school  and  college 
levels.  Nevertheless,  the  component  skills  that  comprise  self-regulated  learning  need  not, 
in  our  opinion,  be  viewed  as  either  exotic  or  as  something  above  and  beyond  "the  basics." 
They  are  in  all  likelihood  the  basic  skills  that  underlie  all  forms  of  successful  learning 
(Resnick  &  Klopfer,  1989).  In  any  event,  given  the  degree  of  success  that  self-regulated 
learners  have  been  reported  to  enjoy,  it  follows  that  understanding  the  behaviors  and 
processes  that  underlie  self-regulated  learning,  as  well  as  designing  instruction  in  ways 
likely  to  facilitate  self-regulation  of  the  learning  process,  represent  important  goals  for 
educational  researchers  and  designers. 

Our  own  research,  at  this  juncture,  has  not  been  primarily  theoretically  motivated. 
However,  after  reviewing  the  literature  surrounding  this  topic,  we  found  it  useful  (and  to 
some  extent,  necessary)  to  impose  an  organizational  structure,  in  the  form  of  a  model,  on 
the  various  and  tangled  dimensions  of  self-regulated  learning  as  reported.  Our  working 
model  of  self-regulated  learning  presently  consists  of  six  dimensions:  Epistemological 
Beliefs,  Metaoognition,  Learning  Strategies,  Motivation/Self-Efficacy,  Contextual 
Sensitivity  and  Environmental  Utilization/Control  (see  Appendix  A  for  a  more  detailed 
description).  Most  of  the  various  self-regulated  strategies  reported  in  the  literature  (for 
example,  see  Pintrich,  Smith  &  McKeachie,  1989;  Weinstein,  Zimmermen  &  Palmer 
1988;  Zimmerman  and  Martinez-Pons,  1986)  fall  into  one  or  another  of  the-  categories  we 
have  constructed.  Contextual  sensitivity,  we  should  note,  although  implicit  in  much  of  the 
published  literature,  is  not  an  area  typically  identified  explicitly  as  an  independent  aspect 
ot  self-regulated  learning.  However,  the  theme  that  cognitive  processes  are  contextually 
bound,  or  "situated"  (Brown,  Collins  &  Duguid,  1989;  Jenkins,  1974;  Rogoff  &  Lave,  1984) 
is  becoming  increasingly  general  in  the  contemporary  literature  on  learning  and ' 
cognition,  particularly  as  it  occurs  in  educational  settings.  We  therefore  decided  to  define 
it  as  a  separate  dimension  in  our  working  model  of  self-regulated  learning. 

In  brief,  in  developing  our  model,  we  reasoned,  following  Zimmerman  (1990),  that 
the  self-regulated  learner  must  be  able  to  both  internally  regulate,  monitor,  evaluate  and 
modify,  when  necessary,  the  learning  process,  and  be  alert  to  and  utilize  or  manage 
contextual  (external)  factors  such  as  course  and  instructor  demands,  where  and  when  to 
study,  who,  when  and  where  to  go  for  assietance,  etc.  Self-regulated  learners  are  possessed 
of  a  belief  system  that  views  knowledge  as  complex  and  evolving,  rather  than  simple  and 
fixed,  and  the  knower  as  capable  of  self-modification.  It  is  also  evident  that  motivational 
factors  mediate  the  utilization  of  both  cognitive  and  environmental  resources  (Borkowski 
Carr,  Rellinger  &  Pressley,  1990).  Individuals  high  in  self-efficacy,  for  example,  are 
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more  likely  to  use  cognitive  and  metacognitive  strategies  and  to  seek  appropriate 
(instrumental)  forms  of  assistance  when  needed  (Karabenick  &  Knapp,  1991;  Schunk 
1991).  At  the  same  time,  there  is  a  positive  relationship  between  a  sense  of  V**™  contro1 
over  learning  outcomes  and  subsequent  motivation  (Dweck,  1989;  Schunk,  1991)  to 
undertake  learning-related  challenges.  Despite  the  many  elements  that  enter  into  it,  there 
is,  as  we  shall  see,  reason  to  believe  that  self-regulated  learning  is  a  unified  process  which 
involves  the  integration  of  appropriate  beliefs  and  utilization  of  cognitive,  metacognitive, 
motivational,  perceptual  and  environmental  components  in  the  successful  resolution  ot 

academic  tasks.  , 

Having  devised  a  working  model  of  self-regulated  learning,  we  set  out  to 
determine  if,  and  to  what  extent,  self-regulated  learning  (thus  defined)  plays  a  significant 
role  in  successful  academic  performance  at  the  college  level.  We  chose  to  do  this  by 
employing  a  self-report  inventory,  of  our  own  design,  composed  of  the  subscales  consistent 
with  our  working  model  discussed  previously.  We  opted  to  develop  such  an  instrument 
because  to  our  knowledge  no  instrument  of  its  kind  existed,  and  we  believed  that  such  an 
instrument  could  prove  valuable  as  a  research  tool  and  would  be  more  efficient  and  cost- 
effective  than  interviewing. 

This  paper  describes  the  development  of  our  questionnaire,  methods  used  in 
collecting  data,  results  of  the  data  analysis,  and  then  discusses  the  degree  which  a 
construct  as  complex  and  multi-faceted  as  self-regulated  learning  can  be  measured  using 
a  self-report  inventory.  In  addition,  the  paper  discusses  some  implications  of  our  working 
model  for  instruction  and  teaching  self-regulated  learning  strategies. 

Method 

Development  of  the  Self-Regulated  Learning  Inventory 

Our  first  step  in  the  creation  of  a  self-regulated  learning  inventory  involved  the 
generation  of  an  item  pool.  We  decided  to  review  the  literature  and  to  construct  our  items 
on  the  basis  of  findings  that  reported  strong  relationships  between  learner-generated 
activities  and  academic  success.  We  then  reviewed  and  analyzed  the  items  eliminating 
those  that  were  too  much  alike  and  rewriting  those  that  were  either  too  complex  or  too  vague. 
This  left  us  with  a  pool  of  seventy-one  items  all  of  which  were  included  in  our  first 
instrument.  Although  the  items  represented  each  of  the  subscales  of  our  working  model 
discussed  previously,  we  decided  to  present  them  randomly  as  a  single  questionnaire.  A 
five  point  Iikert  scale  format  was  chosen  as  most  appropriate  for  this  type  of  instrument. 

The  instrument  included  representative  items  from  each  of  the  subscales  except  the 
epistemological  beliefs  subscale.  Our  first  conceptualization  of  the  model  did  not  include 
the  epistemological  beliefs  dimension,  thus  we  only  wrote  items  for  the  other  five 
subscales.  After  analyzing  the  data  presented  in  this  paper,  however,  we  revised  the 
instrument  and  included  15  items  representing  the  epistemological  beliefs  dimension. 
Most  of  these  items  were  obtained  from  Schommer's  (1990)  work  in  assessing  students 
beliefs  about  the  nature  of  knowledge.  Appendix  B  shows  sample  items  from  each  of  the 
subscales,  including  items  from  the  epistemological  beliefs  subscale  which  we  added  in  the 
revised  version  of  the  instrument  following  the  data  analysis  of  this  study.  The  date 
analyzed  in  the  following  sections  of  this  paper,  however,  only  include  data  collected  with 
the  first  version  of  the  instrument,  which  did  not  include  epistemological  beliefs  items. 

A  pilot  run  was  conducted  to  see  if  directions  were  clear  and  sufficient,  how  long  it 
took  to  respond  to  the  inventory  and  if  the  items  as  written  were  clear  and  comprehensible. 
As  a  result,  a  formal  set  of  instructions  was  composed.  It  was  determined  that  time  to 
complete  the  inventory  ranged  from  20-30  minutes. 
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Subjects 

Our  subjects  were  all  students  enrolled  in  classes  in  the  college  of  education  at  a 
medium  size  mid-Western  university.  Unfortunately,  the  majority  of  education  majors 
continue  to  be  female.  Thus,  our  sample  contains  an  imbalance  in  terms  of  males  (39)  and 
females  (121).  In  terms  of  ethnic  composition,  145  of  our  subjects  were  European  American 
White),  10  were  African  American  (Black),  two  were  Hispanic  American  and  three  were 
Asian  American.  With  respect  to  class  standing,  61  were  sophomores,  58  were  juniors,  and 
35  were  seniors.  Our  sample  also  included  14  graduate  students  and  two  non-degree 
students.  The  mean  age  of  our  subjects  was  22.8  years.  In  total,  the  inventory  was 
responded  to  by  166  students.  Only  160  cases  were  actually  analyzed  due  to  the  failure  of 
some  students  to  properly  report  requested  information  and/pr  respond  to  items. 

Procedure 

The  inventories  were  administered  in  every  instance  by  one  or  the  other  of  the 
authors.  Having  obtained  prior  permission  from  class  instructors,  we  passed  out  the 
inventories  and  read  a  prepared  set  of  instructions.  Classes  ranged  in  size  from  thirty  to 
ten.  Although  participation  was  entirely  voluntary,  no  student  refused  to  fill  out  the 
inventory. 

Results 

We  first  report  on  findings  that  relate  to  the  technical  properties  of  the  inventory. 
Table  1  shows  the  result  of  an  analysis  of  internal  reliability  of  the  five  subscales 
discussed  previously  (MC  represents  the  Metacognition  Scale,  LS  represents  the  Learning 
Strategies  Scale,  MO  represents  the  Motivation/Self-Efficacy  Scale,  CS  represents  the 
Contextual  Sensitivity  Scale,  and  EC  represents  the  Environmental  Utilization/control 
Scale). 

We  are  encouraged  by  these  results,  although  by  no  means  satisfied.  An  analysis 
of  test-retest  reliability,  with  an  eight  week  delay  between  times  of  testing,  revealed  a 
correlation  of  .78,  a  result  we  also  take  to  be  encouraging. 

Table  1 


Reliability  Coefficients  (Cronbach's  Alpha) 


MC 

LS 

MO 

CS 

EC 

Alpha 

.77 

.83 

.77 

.64 

.79 

Our  evidence  with  respect  to  validity  at  this  point  is  mixed.  That  is,  our  items  were 
constructed  on  the  basis  of  findings  in  the  literature  related  to  the  construct  we  set  out  to 
measure.  An  analysis  of  the  correlation  between  scores  on  the  inventory  and  GPA,  our 
measure  of  academic  achievement,  revealed  a  significant  correlation  both  for  the 
mventory  as  a  whole  (represented  as  SRLTOT)  and  for  each  of  the  subscales  (see  Table  2). 
This  result  corresponds  to  findings  as  reported  in  the  supporting  literature  and  provides 
evidence  of  concurrent  validity. 
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Table  2 


GPA  and  Scores  on  the  Inventory 


MC 

LS 

MO 

CS 

EC 

SRLTOT 

Correlation  .41** 

.47** 

.48** 

.31* 

.39** 

.54** 

*  p  <.01,  **  p  <.ooi 

The  result  of  a  factor  analysis  revealed  that  two  factors  account  for  the  largest 
percentage  (30.4)  of  the  variance.  A  general  factor  represented  by  items  from  every 
subscale  (in  all,  52  of  the  71  items)  which  we  labelled  self-regulated  learning  and  a  self- 
efficacy  factor  represented  by  13  of  the  15  items  from  that  subscale.  These  two  items,  as 
noted,  account  for  the  main  portion  of  the  variance.  The  fact  that  a  single  factor  loads 
highest  is  in  line  with  the  findings  of  Zimmerman  &  Martinez-Pons  (1988).  However,  a 
number  (18)  of  other  factors,  small  but  statistically  significant,  also  appeared.  While  this 
complicates  our  ability  to  draw  clear-cut  conclusions  with  respect  to  construct  validity,  we 
found  some  of  these  factors  to  be  suggestive  in  terms  of  potential  areas  of  inquiry  requiring 
further  investigation.  For  example,  it  would  appear  that  some  students  are  instructor- 
based  learners  while  others  are  text-based  learners.  It  would  also  appear  that  further  work 
on  the  inventory  will  need  to  be  undertaken  to  insure  that  its  items  represent  fewer,  and 
more  distinct  factors. 

As  noted,  we  selected  student  GPA  as  our  measure  of  academic  achievement. 
While  scores  on  the  inventory  subscales  in  each  case  correlate  significantly  with  GPA,  the 
largest  correlation  obtained  (see  Table  2)  was  between  GPA  and  total  score  (SRLTOT). 
Analysis  of  the  data  with  respect  to  the  variables  of  class,  age,  and  sex  revealed  significant 
correlations  between  both  sex  (.19  p<  .01)  and  age  (.31  p<.001)  and  total  score  on  the 
inventory  (see  Table  3).  The  correlation  between  class  and  score  on  the  metacognitive 
subscale  was  also  significant  (.21  p<  .01). 

Table  3 

Correlation  Coefficients 


Class  Sex  Age 


MC 

.21* 

.05 

.28** 

LS 

.07 

.21* 

.26** 

MO 

.12 

.19* 

.26** 

CS 

.01 

.12 

.09 

EC 

.08 

.12 

.28** 

SRLTOT 

.13 

.19* 

.31** 

*  p<  .01,  **  p<  .001 
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u     i  ™  the  mean  8COre8  for  malea  811(5  femalea  on  the  inventory  and  its 

wl8C5eJ        ,"a»mum        Possible  for  the  total  inventory  was  355,  which  reports  the 
highest  degree  of  usage  of  self-regulated  learning  strategies.  The  minimum  score 
possible  was  71   The  maximum  and  minimum  scores  for  each  subscale,  respectively  are 
as  follows:  MC:  85  and  17;  LS:  90  and  18;  MO:  75  and  15;  CS:  50  and  10;  and  EC:  55  and  11 

™»fa™ £T5  £  °Ut8COre  male8  on  total  8C0re  88  we»  88  a»  subscales  but 

IS       .         these  differences  are,  in  most  instances,  statistically  significant, 
we  hesitate  in  drawing  any  firm  conclusions  due  to  the  small  number  of  males  in  our 
sample.  We  also  found  as  noted,  a  significant  correlation  (.31,  p<.001)  between  age  and 
*»JZ \ 01de' ,ubJe.cti  tonded  t?  «»»  Wgher.  This  result  is,  in  part,  due  to  the  fact  that 
toe  graduate  students  m  our  sample,  though  few  in  number,  generally  scored  higher  (M  = 
268.1)  on  the  inventory  than  other  groups  (overall  M  =  247.9).  The  fact  that  only 
metecogn.tion  showed  a  significant  correlation  with  class  is  somewhat  misleading.  That 
s,  although  there  were  only  14  graduate  students  in  our  sample,  their  mean  age  was  32 

Lm;zd::!J:ig  only  231  ™us  the  ■™d-to studente  were  ^ the  ^ — 

Table  4 

Sex  X  Inventory  Score  (mean  scores) 


MC 


LS 


MO 


CS 


EC 


SRLTOT 


MALE 
FEMALE 


57.2 
57.8 


N  for  Male=  39 
N  for  Females  121 


61.2 
65.4 


51.4 
54.6 


34.4 
36.2 


35.1 
36.6 


239.3 
250.6 


Discussion  and  Implications  tor  Teaching 
Self-Regulated  Learning  Strategies 

Results  of  the  Inventory 

J2?  reS"ltS  -Pr0Vi(?e  80me  degree  0f  evidence  which  lead  us  to  conclude  both  that  self- 
™Em£S   T"B  W  T  ^P0^8"1  component  in  academic  success  and  that  it  can  be 
measured  with  some  degree  of  validity  and  reliability  via  a  self-report  instrument.  The 
r!™£5        8    ""fX^  mdlCate  a  8ignificant  relationship  exists  between  self- 
2fJll!?  f  &r\mg       ?f     ^  result  is  in  line        P"bli8h8d  ^search  on 

!rmg  (Zrmerman  &  Martinez-Pons,  1986;  1988;  Zimmerman,  1990). 

fn"^  and  GPr^S  7*1 ft T^k W°  al8°  mU>  that  total  score  on  the 

with  th ?Z% %M      T  •*    £  *  8,^ifi,cant  correlation,  there  arose  some  problems 

*  T  £ ,   •  criterion.  For  example,  some  students,  although  not  scoring  high 
in  terns  of  segregation,  appear  nevertheless  to  maintain  a  high  GPA  by  eithe"  avowing 
or  dropping  difficult  courses.  Hence  their  scores  may  actually  have  served  to "lower  tfie 
dSTJ ^dST**  J^-to^  we  w8re  -able  to  obtein  sufficient  tofo^adon  to 
Sin  Uu™,nr  Wh,Ch  8tudentS  actually  dr°PPed  which  and  how  many  courses.  We 

plan  to  gather  this  information  in  a  follow-up  study.  It  should  be  noted  as  well,  that 
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although  the  correlation  between  score  on  the  inventory  and  GPA  was  found  to  be  highly 
significant,  much  of  the  variance  in  student  performance  is  left  unexplained. 

The  fact  that  total  score  showed  the  highest  correlation  with  performance  is  in  line 
with  the  work  of  Zimmerman  &  Martinez-Pons  (1988),  who  also  found  that  self-regulated 
learning  treated  as  a  single,  overarching  factor,  showed  the  strongest  correlation  with 
achievement.  However,  factor  analysis  of  the  data  suggests  that  degree  of  self-regulated 
learning  may  be  mediated  by  "learning  style"  factors  not  yet  clearly  understood.  The  fact 
that  graduate  students  score  highest  suggests  the  greater  presence  of  self-regulated 
learning  in  this  population.  While  this  is  not  entirely  surprising,  the  small  number  of 
graduate  students  in  our  sample  makes  this  finding  suggestive  only.  In  general,  it  can  be 
concluded  that  self-regulation  is  a  significant  element  in  successful  college  stcdent 
performance  and  that  many  students  could  profit  by  forms  of  instruction  that  emphasize 
and  promote  both  the  understanding  and  use  of  the  component  skills  and  attitudes  of  which 
self-regulated  learning  is  comprised. 

Implications  for  Instruction 

Our  research,  as  well  as  the  research  of  a  number  of  others  (Borkowski,  et.  al., 
1990;  Jones  &  Idol,  1990;  Zimmerman  &  Pons,  1986)  lends  support  to  the  claim  that  self- 
regulatory  skills  are  important  components  of  successful  academic  performance.  An 
important  question  is:  what,  if  any,  implications  do  such  findings  have  for  instruction? 
Before  addressing  this  question,  we  would  like  to  underscore  the  importance  of  the  issue  by 
noting  that  classroom  instruction,  even  at  the  university  level,  may  not  only  fail  to  promote 
self-regulated  learning,  but  actually  suppress  it  (McCaslin  &  Good,  1992;  Farnham- 
Diggory,  1990).  While  self-regulation  is  important  for  superior  academic  performance, 
academic  conditions  do  not  necessarily  promote  either  the  use  or  development  of  this 
ability.  Having  said  this,  we  think  at  least  two  implications  can  be  drawn. 

First,  the  fact  that  self-regulated  learning  plays  a  crucial  role  in  academic  success 
indicates  that  teachers  need  to  assess  and  take  into  account  this  dimension  of  the  learning 
process  when  considering  the  classroom  performance  of  particular  students.  Academic 
difficulty  may  be  less  a  matter  of  ability  than  a  failure  of  students  to  (know  how  to)  take 
control  of  the  learning  process  to  a  sufficient  degree.  More  specifically,  the  problem  may 
lie  in  one  or  several  areas  directly  tied  to  self-regulation  of  the  learning  process.  For 
example,  lack  of  metacognitive  awareness  and  self-monitoring  may  lead  to  failure  to 
apply  the  skills  the  learner  possesses  under  conditions  where  they  clearly  apply  (Schunk, 
1991).  On  the  other  hand,  the  learner  may  fail  to  acquire  an  understanding  of  the 
connection  between  specific  learning  tactics  and  specific  learning  situations  and 
outcomes  (Pressley,  et.  al.,  1990).  Some  learners  may  interpret  challenging  classroom 
tasks  as  potential  sources  of  negative  evaluation  of  their  competence,  hence  fail  to  apply  the 
knowledge  and  skills  they  possess  (Dweck,  1989).  To  overcome  such  deficiencies,  it  is 
necessary  that  learners  be  provided  with  information  which  ties  strategy  use  to  specific 
learning  outcomes  (Pressley,  et.  al.,  1990).  Such  information  is  also  likely  to  provide 
feedback  which  links  learning  outcomes  to  specific  student  generated  activities,  a  fact 
which  is  known  to  affect  self-efficacy  attributions  (Schunk,  1991). 

A  significant  contribution  of  our  working  model  is  that  it  illustrates  the 
comprehensive  nature  of  self-regulated  learning  and  the  importance  of  the  critical 
relationships  among  its  components.  With  a  comprehensive  understanding  of  the 
different  dimensions  of  self-regulated  learning,  teachers  can  assess  the  particular  areas 
in  which  students  may  be  deficient  and  help  to  remediate  these  weaknesses.  For  example, 
a  teacher  may  assess  that  a  student  has  excellent  metacognitive  strategies,  such  as 
reflecting  and  self-monitoring,  and  also  a  good  repertoire  of  learning  strategies. 
However,  if  the  student  believes  that  the  ability  to  learn  is  innate  rather  than  acquired, 
he/she  may  choose  not  to  engage  in  a  learning  situation  because  he/she  doesn't  believe  that 
he/she  can  acquire  the  required  ability  to  solve  the  learning  problem,  thus  never  using  his 
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or  her  excellent  cognitive  strategies.  Such  relationships  and  interactions  are  not 
identified  in  many  of  the  traditional  study  skills  models,  courses  or  programs. 

Second,  it  may  be  necessary,  particularly  when  self-regulatory  skills  are  found  to 
be  lacking,  to  develop  instruction  specifically  aimed  to  counteract  this  deficiency.  In  fact, 
we  believe  that  since  self-regulation  empowers  students,  it  should  always  be  a  component  of 
the  curriculum.  In  this  regard,  our  thinking  is  in  line  with  an  instructional  approach 
referred  to  as  cognitive  apprenticeship  (Rogoff,  1990).  As  Rogoff  (1990,  p.39)  notes, 
cognitive  apprenticeship  occurs  when  "active  novices  advance  their  skills  and 
understanding  through  participation  with  more  skilled  partners  in  culturally  organized 
activities."  Cognitive  apprenticeship  is,  in  other  words,  a  form  of  socially  mediated 
instruction  wherein  (1)  to-be-learned  skills  are  modeled  by  a  more  experienced  "expert" 
[adult  or  peer],  (2)  made  explicit  by  the  "expert"  through  think-aloud  demonstrations  in  the 
application  and  regulation  of  the  component  skills  and  (3)  over  the  course  of  learning  the 
"novice"  is  induced  to  accept  increasing  responsibility  for  hi8^er  performance  of  the 
target  skill  (Englert  &  Raphael,  1989).  Also  crucial  to  the  cognitive  apprenticeship  model 
is  the  notion  that  socially  mediated  learning  is  most  effective  when  occurring  within  the 
"zone  of  proximal  development"  (Rogoff,  1990).  That  is,  such  instruction  attempts  to  enter 
a  student's  optimal  region  of  sensitivity  to  social  guidance  in  order  to  facilitate  cognitive 
growth.  In  order  to  accomplish  this  goal,  one  needs  to  assess  the  learner's  readiness  to 
benefit  from  a  particular  instructional  intervention.  Assessing  the  ability  to  self-regulate 
the  learning  process  would  appear,  to  us,  to  be  an  important  qualifying  criterion. 

Our  inventory  could  be  used  as  a  diagnostic  tool  to  identify  the  specific  areas  in 
which  students  have  difficulty  in  self-regulating  the  learning  process.  The  information 
provided  by  the  inventory  could  be  used  by  a  taacher  as  a  basis  for  planning  or  developing 
individual  instructional  prescriptions  for  both  remediation  and  enrichment.  In  addition, 
such  information  could  be  valuable  for  determining  the  degree  to  which  a  particular 
learner  is  prepared  to  benefit  from  self-instructional  materials,  including  many  forms  of 
computer-based  instruction.  Many  students  who  flunk  or  never  complete  a  self- 
instructional  course  or  program  fail  because  they  lack  the  necessary  ability  to  self- 
regulate  the  learning  process.  Perhaps  one  approach  for  reducing  the  failure  rate  of  self- 
instructional  courses  or  programs  would  be  to  administer  the  instrument  to  students  before 
taking  the  course  and  then  adjusting  the  program  or  advise  students  in  accordance  with 
performance  on  the  inventory. 

Many  models  for  the  design  of  instruction  advocate  the  assessment  of  the  learner's 
entry  behaviors  and  learner  characteristics.  However,  many  of  these  models  only 
advocate  assessing  the  domain-related  knowledge  which  the  learner  brings  to  the 
learning  task.  Our  working  model  suggests  that  educators  and  instructional  designers 
should  not  only  assess  the  learner's  knowledge  base  and  skills,  but  should  also  assess 
information  about  the  student's  epistemological  beliefs,  motivational  level,  use  of 
metacognitive  and  learning  strategies,  level  of  contextual  sensitivity,  and  ability  to 
control  and  utilize  his  or  her  learning  environment.  Given  the  fact  that  students  enter  the 
learning  process  at  varying  degrees  of  self-regulation  suggests  that,  ideally,  a  variety  of 
instructional  options  should  be  developed  to  suit  the  needs  of  different  types  of  learners. 

The  Role  of  Interactive  Videodisc-Based  Technologies 

Considering  the  evidence  that  self-regulated  learning  is  an  important  component 
m  academic  success,  the  next  logic  question  is:  What  is  the  best  way  to  teach  these  higher- 
order,  complex,  thinking  strategies  and  skills?  Stated  more  specifically,  what  are  the  most 
effective  instructional  strategies  for  promoting  both  the  understanding  and  consistent  use 
of  self-regulated  learning  strategies? 

We  propose  that  an  effective  and  efficient  means  of  tackling  the  problem  involves 
the  creative  and  resourceful  use  of  interactive  videodisc-based  technology  along  the  lines 
of  the  "anchored  instruction"  strategy  of  the  Cognition  and  Technology  Group  at 
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Vanderbilt  University  (Cognition  &  Technology  Group  at  Vanderbilt,  1990).   The  basic 
premise  of  what  the  Vanderbilt  group  refers  to  as  anchored  instruction  is  that  learning  is 
most  natural  and  most  viable  when  it  is  "situated"  in  realistic  environment*  that  permit 
"sustained  exploration  by  students  and  teachers  and  enable  them  to  understand  the  kinds 
of  problems  and  opportunities  that  experts  in  various  areas  encounter  and  the  knowledge 
that  these  experts  use  as  tools"  (p.  3).  They  (drawing  upon  the  recent  work  of  Allan  Collins 
and  John  Seely  Brown  [Brown,  Collins  &  Duguid,  1989],  who  have  been  instrumental  in 
advancing  the  ideas  of  situated  cognition  and  cognitive  apprenticeships)  argue  that 
situated  cognition  provides  a  broad,  useful  framework  that  emphasizes  the  importance  of 
focusing  on  everyday  cognition,  authentic  tasks,  and  the  value  of  in -context  apprenticeship 
training.  As  Brown,  et.al.  (1989)  have  noted,  natural  learning  proceeds  most  effectively 
in  the  context  of  authentic  (rooted  in  the  realistic  practices  that  prevail  in  a  particular 
culture)  tasks.  However,  given  the  structure  of  our  contemporary  educational  system, 
authentic  contexts  of  apprenticeship  for  the  acquisition  of  the  complex  intellectual  skills 
necessary  for  success  in  higher  education  are  difficult  to  create  in  a  cost-effective  and 
efficient  manner.  Clever  use  of  interactive  videodisc-based  technology,  however,  may 
provide  the  key  for  making  preparatory  educational  experiences  more  authentic. 

According  to  the  Vanderbilt  group  (Cognition  &  Technology  Group  at  Vanderbilt, 
1990),  anchoring  instruction  in  videodisc-based  problem  solving  environments  has 
several  distinct  advantages.  First  and  foremost,  it  makes  the  "idea  of  transforming  school 
instruction  into  apprenticeships  more  feasible"  (p.  8).  It  is,  in  other  words,  more  realistic 
to  ground  (anchor)  problem  solving-based  instruction  in  the  simulated  reality  of  a 
videodisc  than  to  place  classes  full  of  students  into  authentic,  real  world  conditions  that 
require  problem  solving  (e.g.,  planning,  navigating  and  undertaking  a  journey  by  boat, 
etc.).  Videodisc-based  contexts  also  have  the  advantage  of  compressing  what  would  take 
days,  perhaps  weeks  and  months  in  the  real  world,  into  minutes  and  hours  in  the 
classroom,  as  well  as  making  it  possible  for  students  to  revisit  event  segments  and  test 
their  memories  against  actual  aspects  of  events,  something  not  generally  possible  in  real 
life.  In  short,  employing  videodisc  technology  holds  the  potential  for  making  classroom 
learning  more  authentic  and  apprentice  like. 

The  Cognition  and  Technology  Group  at  Vanderbilt  (1990)  have,  to  date,  employed 
the  notion  of  anchored  instruction  within  two  key  projects.  Preliminary  findings  show 
very  positive  results  indicating  that  students  so  instructed  are  more  likely  to  employ 
higher-order  thinking  than  comparison  groups  receiving  more  traditional  forms  of 
instruction,  However,  the  Vanderbilt  group  has  focused  largely  on  elemental^  school  age 
children  and  problem  solving  skills  that  might  best  be  characterized  as  deductive  in 
nature. 

Our  own  interest  lies  with  older,  college-age  students.  Furthermore,  the  kinds  of 
higher-order  thinking  skills  we  wish  to  facilitate  are  best  characterized  as  self-regulatory 
skills.  Nevertheless,  we  believe  that  the  anchored  instruction  approach  using  interactive 
videodisc-based  technology  is  a  very  promising,  viable  approach  for  promoting  self- 
regulated  learning  strategies. 

Design  Considerations  of  Teaching  Self-Regulated  Learning  Strategies 

We  are  currently  in  the  process  of  designing  instruction  (using  interactive 
videodisc-based  technologies)  for  promoting  self-regulated  learning  strategies  in  college 
students.  As  a  part  of  this  effort,  we  are  conducting  a  study  to  determine  the  most 
appropriate  and  effective  instructional  strategy  (or  strategies)  for  teaching  these  higher- 
order  thinking  skills.  The  instructional  strategy,  or  strategies,  best  suited  to  promote  a 
more  general  acquisition  of  such  skills  remains  largely  an  empirical  question.  Three 
such  strategies  can  be  identified  in  the  literature:  the  stand-alone  strategy,  the  embedding 
strategy  and  the  immersion  strategy  (Ennis,  1989;  Prawat,  1991). 
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The  stand-alone  strategy  assumes  that  there  exists  a  set  of  general,  content 
independent  cognitive  skills,  which  can  and  should  be  taught  as  such  (Feuerstein,  1980).  It 
is,  say  its  proponents,  up  to  the  educational  community  to  make  room  in  the  curriculum  for 
such  instruction.  Critics  charge,  however,  that  explicitly  taught  generic  thinking  skills 
typically  fail  to  transfer  either  across  the  curriculum  or  to  the  work  place  (Larkin,  1989). 
The  embedding  strategy  is  built  on  the  premise  that  higher-order  thinking  skills  are  best 
taught  explicitly  as  an  integral  component  of  content-baaed  instruction  (Beyer,  1987; 
Ennis,  1989).  Those  who  prefer  an  embedding  strategy  also  assume  that  thinking  skills 
are,  to  a  significant  extent,  context  bound.  From  this  perspective,  the  contemporary 
curriculum  need  only  be  slightly  modified  in  order  to  incorporate  instruction  that 
emphasizes  the  how  of  learning  as  much  as  the  what  of  learning.  To  critics,  however,  the 
feasibility  of  embedding  an  emphasis  on  higher  level  thinking  skills  at  no  cost  to  the 
standard  curriculum  appears  questionable  (Prawat,  1991).  Furthermore,  just  as  with  the 
stand-alone  strategy,  the  problem  of  cross-content  transfer  also  plagues  the  embedding 
approach. 

A  third,  less  common  and  less  well-defined  approach  to  promoting  the  development 
of  higher  level  thinking  is  represented  by  the  immersion  approach.  Essentially,  those  who 
promote  an  immersion  strategy  argue  that  higher  level  thinking  will  emerge  naturally 
when  students'  own  ideas  are  taken  seriously,  and  as  they  are  immersed  in  the  main  ideas 
and  issues  that  define  a  particular  field  of  study  (Prawat,  1991).  The  immersion  approach 
is  a  content  first  strategy.  However,  the  emphasis  is  more  on  the  importance  of  ideas  as 
tools  for  unlocking  content  rather  than  on  either  the  content  per  se,  or  the  cognitive 
processes  (independently  construed)  applied  to  that  content.  Importantly,  it  is  thought  by  its 
proponents  that  cross  content  transfer  of  the  thinking  skills  emerging  out  of  students' 
confronting  the  main  ideas  that  drive  a  discipline  is  more  likely  with  the  immersion 
approach  because  they  arise  implicitly  and  naturally  (rather  than  explicitly  and 
artificially)  in  the  process. 

It  can  be  seen  that  proposals  for  the  remediation  of  the  failure  of  education  to 
promote  the  development  of  higher-order  thinking  are  not  lacking.  Unfortunately,  as 
Prawat  (1991,  p.  8)  in  his  review  of  the  research  literature  notes,  "few  studies  to  date  have 
attempted  to  compare  the  relative  effectiveness  of  different  ways  of  promoting  higher-order 
thinking  in  students." 

We  are  currently  in  the  process  of  conducting  a  study  which  uiei  each  of  the  three 
approaches  previously  described  to  teach  self-regulated  learning  strategies.  The  intent  of 
the  design  has  been  to  determine  which  approach  (the  stand  alone,  embedding,  or 
immersion)  best  facilitates  the  development  of  higher-order,  specifically  self-regulating, 
thinking  skills  when  employing  a  videodisc-based  instructional  system.  It  is  our  hunch 
that  the  immersion  approach,  by  capitalizing  on  natural  forms  of  learning  when  presented 
within  the  framework  of  the  cognitive  apprenticeship  model,  may  prove  the  more  powerful 
method  of  the  three.  Ultimately,  based  on  the  results  of  this  and  future  studies,  we  hope  to 
continue  this  line  of  inquiry  by  developing  an  intelligent  tutoring  system  using 
interactive  videodisc-based  instruction  designed  to  teach  self-rogulated  learning 
strategies  to  college-age  students. 
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Appendix  A 
Dimensions  of  Self-Regulated  Learning 
A  Working  Model 

General  definition:  (A)  The  ability  to  monitor,  regulate,  evaluate,  sustain,  and 
strategically  modify,  when  necessary,  the  learning  process  and  (B)  sensitivity  to  and 
ability  to  exercise  control  over,  motivational  and  contextual  factors  that  affect  learning 
outeomes  The  basic  components  of  self-regulated  learning  include  (1)  epistemological 
beliefs  (2)  motivational  processes  (3)  metacognitive  processes  (4)  learning  strategies  (5) 
contextual  sensitivity  and  (6)  environmental  control  and/or  utilization.  Self-regulated 
learners  are  possessed  of  a  belief  system  that  views  knowledge  as  complex  and  evolving 
rather  than  simple  and  fixed,  and  the  knower  as  capable  of  self-modification  An 
mdmdual  is  a  self-regulated  learner  to  the  degree  that  she/he  is  able  to  effectively  monitor 
and  regulate  (control)  and  sustain  the  learning  process,  apply  a  variety  of  appropriate  and 
efficient  strategies  to  learning  problems  encountered,  maintain  a  sense  of  competence, 
(intrinsic)  motivation  and  personal  agency,  accurately  diagnose  the  character  and 
demands  of  particular  learning  challenges,  and  effectively  utilize  and  control 
environmental  factors  that  have  a  bearing  on  learning  outcomes. 

Six  Dimensions  of  Self-Regulated  Learning 

A.  Epistemological  Beliefs:  Defined  as  relatively  enduring  and  unconscious  beliefs 
about  the  nature  of  knowledge  and  the  process  of  knowing. 

B.  Motivation:  Refers  to  goal-oriented  effort  that  is  a  complex  function  of  goal  value,  goal 
accessibility,  perceived  likelihood  of  success  and  one's  sense  of  self-efficacy. 

C  Metacognition:  Defined  generally  as  (1)  knowledge  about  cognition  and  (2) 

awareness  and  conscious  regulation  of  one's  thinking  and  learning.  The  executive 
engine  of  cognition. 


D 


Naming  Strategies:  Refers  to  both  operative  knowledge  of  specific  learning  tactics 
(highlighting,  summarizing,  etc.)  and  the  ability  to  combine  various  tactics  into  an 
effective  learning  plan. 

E.  Contextual  Sensitivity:  Refers  to  the  ability  to  "read"  the  learning  context  for  what  it 
specifies  regarding  the  demands  of  a  particular  problem  setting  and  what  it  affords  in 
the  way  of  problem  resolution. 

F.  Environmental  Utilization  I  Control:  Refers  to  the  utilization  and  management  of 
circumstan  nd  resources  external  to  the  self  in  the  pursuit  of  learning-related 
goals. 


Appendix  A  (Continued) 
Graphical  Representation  of  the  Working  Model 


13 


Self-Regulated  Learning 


Epistemological 
Beliefs 


Metacognition 


Environmental 
Utilization/Control 


Performance 


653 


Appendix  B:  Subscales  of  the  Self-Regulated 
Learning  Inventory  with  Sample  Items 
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Circle  One 
CLASS:  F  S  Jr  Sr  Gr 
SEX:   M  F 

ETHNICITY:  EA  AA  HA 
HIGH  SCHOOL:  U   S  R 


SELF-REGULATED  LEARNING  INVENTORY  9/17/92 

©  Lindner  &  Harris,  V  3.0 


Other 


ASA    NA  Other. 


GPA  (on  4.0  scale): 
AGE:  


TEL  (last  4  digits): 


INSTRUCTIONS:  Please  read  each  statement  and  then  circle  a  response  according  to  the  following  key: 
a  =  Almost  always  typical  of  me 
b  =  Frequently  typical  of  me 
c  =  Somewhat  typical  of  me 
d  =  Not  very  typical  of  me 
e  =  Not  at  all  typical  ot  me 

"J*^  ™d  ™«^te,y  ■»  Possible  by  selecting  the  response  most  descriptive  of  your  usual  approach,  anchor  attitude 
!T2S^  T^r*'  Tfyl0nUe  yoursclf  t*™^*  to  how  well  the  statement  describes  you,  notm  SoThow  2  d*k 
you  should  be  or  what  others  think  of  you.  There  arc  no  right  or  wrong  answers.  Your  responses  will  be  kept "  o^SSid 
are  for  research  purposes  only.  Please  complete  all  the  items.  P        y  *™ 


Not  at  all  typical  of  me 

Not  very  typical  of  me  

Somewhat  typical  of  me  

Frequently  typical  of  me  

Almost  always  typical  of  me  


Sample  Metacognitive  Items 

1 .  After  reading  new  information  for  a  class,  I 
mentally  review  it  to  get  a  sense  of  how 
much  I  have  remembered. 

2.  When  studying,  I  make  a  mental  note  of 
concepts,  terms  or  ideas  I  don't  fully 
understand. 


£  ample  Learning  Strategy  Items 

3.  A  study  strategy  I  use  to  memorize  a  list  of 
Wvcral  things  is  to  recite  and  rehearse  the 
items  until  I  can  recall  them  from  memory. 

4.  When  I  have  to  learn  unfamiliar  concepts  or 
ideas  which  are  related,  I  use  mental  images 
to  help  tie  them  together. 

Sample  Motivational  Items 

5.  Mastery  of  new  knowledge  or  skills  is  more 
important  to  me  than  how  well  I  do  com- 
pared to  others. 

6.  I  fiud  that  if  I  don'!  expect  to  do  well  in  a 
^u-ii-t.  1  become  less  motivated. 


Not  at  all  typical  of  me 

Not  very  typical  of  me  

Somewhat  typical  of  me  

Frequently  typical  of  me  

Almost  always  typical  of  me  


a  b  c  d  e 


Sample  Contextual  Sensitivity  Items 

7.  No  matter  what  kind  of  exam  I  am 

preparing  for,  I  always  use  the  same  study 
techniques. 


8.  I  adapt  the  study  strategies  I  use  based  on 
a  5  c  d  e         the  type  and  demands  of  a  particular 

course. 

Sample  Environmental  Control  Items 

9.  If  I  do  not  understand  the  material 
presented  either  in  the  text  or  lecture,  I  try 
to  get  help  from  someone  who  does. 

10.  When  studying,  I  isolate  myself  from 
anything  that  might  distract  me. 


a  b  c  d  e 


a  b  c  d  e 


a  b  c  d  e 


a  b  c  d  c 


a  b  c  d  e 


a  b  c  d  e 


a  b  c  d  e 


a  b  c  d  c 


Sample  Epistemological  Beliefs  Items 

1 1.  Really  smart  sludents  don't  have 
to  work  hard  to  do  well  in  school. 


12.  If  a  person  tries  too  hard  to  under- 
stand a  problem,  they  will  most 
likely  just  end  up  being  confused. 
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Abstract 

This  study  investigated  the  reading  preferences  of  American  Indian  and  Anglo 
children  with  respect  to  ethnicity  of  main  character,  gender  of  main  character,  and 
setting  in  children's  stories.  The  160  subjects  consisted  of  80  American  Indians  and  80 
Anglos  from  Grades  2  and  3,  divided  evenly  by  gender  and  grade  level.  The  study 
yielded  significant  preference  differences  for  ethnicity  of  main  characters,  gender  of 
characters,  and  type  of  setting  Both  Indian  children  and  Anglo  children  preferred  to 
read  about  Indian  characters  over  Anglo  characters.  A  majority  of  all  the  subjects 
indicated  a  preference  to  read  stories  with  both  an  American  Indian  and  an  Anglo  as 
main  characters,  rather  than  stories  with  two  Indians  or  two  Anglos  as  main 
characters.  Males  preferred  to  read  stories  wim  at  least  one  male  character  over 
stories  with  only  females;  whereas,  girls  preferred  to  read  about  two  boys  instead  of  a 
boy  and  a  girl.  Both  Indian  and  Anglo  subjects  preferred  stories  set  in  a  suburban-type 
setting  over  reservation-like  settings. 
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Introduction 

Educational  reform  is  of  prime  concern  and  interest  to  educators  as  well  as  the 
general  public.  This  concern  is  understandable  when  13%  of  all  17-year-olds,  and  40% 
of  all  minority  youth  can  be  considered  functionally  illiterate,  (National  Commission 
on  Excellence  in  Education,  1983).  The  concern  still  exists  for  students  who  proceed  on  to 
attend  college.  Approximately  25%  of  the  Hispanic  and  American  Indian  students 
attending  one  of  the  nation's  largest  universities  report  that  they  did  not  feel 
adequately  prepared  for  their  coursework,  (Morrison  Institute,  1990).  These  problems 
and  others  are  being  addressed  by  different  educational  reform  measures.  One  focus  of 
the  reform  measures  is  to  decrease  illiteracy  by  improving  reading  achievement  in  the 
primary  grades. 

One  way  of  improving  reading  achievement  is  to  provide  students  with 
interesting  material.  Cecil's  (1984)  study  revealed  that  interest  in  reading  material 
has  a  positive  relationship  to  comprehension  of  the  content.  Research  also  indicates 
that  intrinsic  motivation  to  read  increases  when  children  are  exposed  to  materials  that 
reflect  their  reading  preferences  (Busch,  1972).  Clearly,  authors  and  publishers  should 
use  children's  preferences  as  an  important  basis  when  designing  stories  and  other 
educational  materials  for  children  (Harper-Marinick,  Sullivan,  and  Igoe,  1990; 
Summers  and  Lukasevich,  1983).  _ 

This  study  investigated  children's  preferences  related  to  three  variables  in 
stories:  ethnicity  of  the  main  characters,  gender  of  the  main  characters,  and  setting  of 
story.  These  variables  are  particularly  relevant  to  current  concerns  about 
multiculturalism  and  equity  in  our  society. 

Children's  preference  for  ethnicity  of  the  main  character  is  a  component  ot 
reading  material  that  has  been  neglected.  Studies  have  shown  that  primary  grade 
children  exhibit  a  preference  for  their  own  ethnicity  when  making  friendship  choices 
(Braha  &  Rutter.  1980;  Davey  &  Mullin,  1982;  Denscombe,  1983).  This  suggests  that 
children  may  also  prefer  to  read  stories  with  characters  of  their  own  ethnicity. 
However  the  availability  of  reading  material  about  children  of  their  own  ethnicity  is 
limited  for  many  minority  children.  In  a  random  sample  of  203  stories  found  in  1st,  3rd 
and  5th  grade  basal  readers,  only  one  story  containing  an  American  Indian  was  found  at 
the  first  grade  level,  six  at  third,  and  nine  at  fifth  (Reyhner,  1986). 

Research  on  gender  of  the  main  character  suggests  that  children  in  the  primary 
gTade  children  prefe/to  read  stories  about  characters  of  their  same  sex,  with  boys 
demonstrating  a  stronger  same-sex  preference  than  girls.  Rose,  Zimet  and  Blom  (1972) 
found  that  first-grade  boys  preferred  stories  with  male  characters  and  first-grade  girls 
preferred  stories  with  female  characters.  Research  by  Harper-Mannick,  et  al.  (1990) 
Tevealed  that  first-  and  third-grade  Anglo,  Hispanic,  and  Black  inner-city  students 
strongly  preferred  both  same-sex  and  combined-sex  (a  boy  and  a  girl)  pair  of  characters 
over  those  of  the  opposite  sex.  Beyard-Tyler  and  Sullivan's  (1972)  study  revealed  that 
among  adolescents  in  gTades  7,  9,  and  11,  boys  preferred  to  read  stories  with  male 
protagonists  and  girls  preferred  to  read  stories  with  female  protagonists.  However, 
both  boys'  and  girls'  preferences  for  males  increased  at  the  highest  grade  level. 

The  setting  of  children's  stories  has  been  an  area  of  increased  interest  in  recent 
years.  It  is  a  popular  aduU  belief  that  inner-city,  minority  children  should  be  exposed 
to  reading  materials  that  reflect  their  everyday  environment.  However,  research  by 
Okada  and  Sullivan  (1971)  and  Harper-Marinick,  et  al.  (1990)  indicate  that  this  belief 
may  not  be  consistent  with  children's  preferences.  Findings  from  their  studies  indicate 
that  inner-city  children  prefer  stories  with  a  suburban- type  setting. 

The  present  research  was  conducted  to  study  the  reading  preferences  of  Anglo 
and  American  Indian  second  and  third  grade  children  with  respect  to  ethnicity  of  main 
character,  gender  of  main  character,  and  story  settmg.  The  study  investigated 
p,eferences  for  reading  stories  K'ith  either:  1)  Anglo  main  characters,  American  Indian 
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main  characters  or  both,  2)  male  main  characters,  female  main  characters,  or  both,  and 
3)  settings  similar  to  or  different  from  the  surroundings  in  which  the  subjects  live. 
Preferences  were  investigated  across  ethnicity  and  gender  of  the  students. 

Method 

Subjects 

A  total  of  160  children,  80  Anglo  and  80  American  Indians  from  grades  two  and 
three  served  as  subjects.  Eighty  subjects  were  male  and  80  were  female;  81  were  in  the 
third  grade  and  79  in  second  grade.  The  Anglo  participants  came  from  a  predominantly 
Anglo  elementary  school  located  in  a  Southwest  metropolitan  area.  The  American 
Indian  participants  came  from  two  elementary  schools  located  on  Indian  reservations 
near  the  same  metropolitan  area.  The  socioeconomic  status  of  the  students  on  the 
reservations  is  similar  to  many  Southwest  minority  populations.  The  income  level, 
unemployment  rate,  and  school  drcpout  rate  are  such  that  many  of  the  students  on  the 
reservation  are  generally  considered  to  be  "at  risk"  of  academic  failure. 
Materials 

The  materials  consisted  of  18  black  and  white  81/2"  X  11"  illustrations 
adapted  from  previous  reading  preference  studies  (Okada  &  Sullivan,  1971;  Harper- 
Marinick,  et  ah,  1990).  The  illustrations  were  organized  into  three  categories: 
ethnicity  of  main  character,  gender  of  main  character,  and  story-setting.  Each  category 
consisted  of  three  pairs  of  illustrations. 

Materials  for  ethnicity  of  main  character  included  three  pairs  of  illustrations 
representing  the  following  combinations  of  children  reading  a  book  at  a  table:  1)  two 
American  Indians,  2)  two  Anglos,  and  3)  an  Anglo  and  American  Indian.  This  category 
was  designed  for  use  on  a  same-sex  basis,  that  is,  materials  picturing  only  males  were 
used  with  male  subjects  and  materials  showing  only  females  were  used  with  female 
subjects.  Body  positions  and  physical  surroundings  were  consistent  from  picture  to 
picture.  Ethnicity  was  depicted  by  slight  changes  in  physical  features,  such  as  skin 
tone,  dress,  and  length  and  style  of  hair.  The  Indian  characteristics  in  the  illustrations 
were  based  on  recommendations  from  American  Indian  artists. 

The  characters  used  in  the  ethnicity  illustrations  were  also  used  for  the  gender 
of  main  character  illustrations.  They  were  arranged  to  include  the  following 
combinations  of  children  reading  a  book:  1)  two  boys,  2)  two  girls,  and  3)  a  boy  and  a 
girl.  These  materials  were  designed  for  use  on  a  wi thin-ethnicity  basis.  Materials 
picturing  only  Anglos  were  used  with  Anglo  children  and  materials  showing  only 
American  Indian  children  were  used  with  American  Indian  children. 

The  story-setting  materials  included  three  illustrations  depicting 
environmental  settings  commonly  used  in  children's  stories:  1)  a  school  exterior,  2)  a 
home  exterior,  and  3)  a  shopping  area.  These  illustrations  were  drawn  from 
photographs  of  actual  settings.  Two  illustrations  were  prepared  for  each  of  the 
settings,  one  depicting  a  rniddle-class  metropolitan  setting  and  the  other  depicting  a 
setting  representative  of  an  Indian  reservation  in  the  Southwest. 

An  example  pair  of  illustrations  for  each  of  the  three  variables  (ethnicity  of 
main  character,  gender  of  main  character,  and  story  setting)  is  shown  in  Figure  1. 


Insert  Figure  1  about  here 


Procedures 

Each  subject  was  interviewed  individually  by  one  of  the  two  trained 
experimenters.  The  subjects  were  given  the  following  directions; 

"I  have  a  friend  who  writes  stories  for  children  just  like  you.  My 
friend  wants  to  write  a  story  about  these  pictures  but  dcesn't  know 
which  ones  you  would  most  like  to  read  about.  Look  carefully  at 
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these  two  pictures  and  tell  me  which  one  you  would  rather  read  about." 
The  experimenters  pointed  to  each  illustration  and  described  it  orally  in  words  that 
noted  the  relevant  contrast  in  the  pair  (e.g.,  "Which  would  you  rather  read  about,  the 
two  Indian  children  in  this  picture,  or  the  two  white  children  in  this  picture?"). 
Experimenters  accepted  either  a  verbal  response  or  the  child  pointing  to  the 
illustration  of  his  or  her  choice.  Left-to-right  positioning  was  systematically 
alternated  for  each  pair  of  pictures  to  negate  the  possibility  that  position  preferences 
would  influence  the  results.  Each  experimenter  ran  approximately  the  same  number  of 
male  and  female  subjects  from  both  gTade  levels  and  across  both  ethnic  groups. 

Each  subject  was  shown  nine  pairs  of  illustrations,  three  for  ethnicity  of 
characters,  three  for  gender  of  characters,  and  three  for  setting.  For  example,  for 
ethnicity  the  three  illustrations  portrayed  two  Anglo  children  together,  two  American 
Indian  children  together,  and  one  Anglo  and  one  Indian  child  together.  For  each  pair, 
the  child  was  asked  to  select  the  illustration  s/he  would  prefer  to  read  about.  Similar 
procedures  were  employed  for  the  three  illustrations  for  gender  (two  boys,  two  girls,  boy 
and  girl),  and  the  three  for  setting  (three  different  suburban  settings  paired  with  three 
different  reservation-like  settings).  The  experimenters  recorded  responses  on  prepared 
forms. 

Data  Analysis 

The  proportion  of  times  that  each  illustration  type  (e.g.  same  ethnicity,  both 
ethnicities,  opposite  ethnicities)  was  chosen  over  the  other  two  was  recorded  for 
ethnicity  and  for  gender  characters.  Three  separate  chi-square  analyses  were 
employed  to  analyze  each  contrast  (same  vs.  opposite,  same  vs.  both,  and  both  vs. 
opposite)  by  ethni  ity  and  gender  of  the  subjects. 

The  story  setting  responses  were  scored  I  for  suburban  settings  and  0  for 
reservation-like  settings.  A  score  of  3  indicates  that  the  subject  chose  all  three  of  the 
suburban  settings  over  the  reservation-like  settings.  Analysis  of  variance  (ANOVA) 
was  used  for  story  setting  data  to  analyze  differences  among  subjects  by  ethnicity  and 
gender. 

Data  in  all  three  categories  were  collapsed  across  grade  levels  because  the 
scores  of  second-graders  and  third-graders  were  very  similar  on  each  of  the  three 
variables  and  no  grade-level  interactions  were  present. 

Results 

The  results  are  reported  below  by  ethnicity  of  main  character,  gender  of  main 
character,  and  story  setting. 
Ethnicity  of  Main  Character 

Table  1  shows  the  proportion  of  subjects  who  chose  characters  of  the  same 
ethnicity,  both  ethnicities,  and  the  opposite  ethnicity  in  the  three  pairs  of 


Insert  Table  1  about  here 


illustrations,  It  can  be  seen  that  both  American  Indian  and  Anglo  children  consistently 
preferred  illustrations  with  an  American  Indian  and  an  Anglo  over  those  with  two 
children  of  their  same  ethnicity  and  over  those  with  two  children  of  the  opposite 
ethnicity.  American  Indians  chose  both  ethnicities  over  American  Indians  only  in  70% 
of  these  pairings,  and  both  over  Anglos  only  in  53%  of  the  pairings.  Anglos  chose  both 
over  Anglos  only  in  65%  of  the  pairings  and  both  over  American  Indians  only  in  66%. 
Overall,  illustrations  showing  both  ethnicities  were  selected  63%  of  the  time  when 
paired  with  the  other  two  choices.  The  opposite  ethnicity  was  selected  in  44%  of  the 
pairings,  and  the  same  ethnicity  was  chosen  in  43%  of  its  pairings. 
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The  same  vs.  opposite  ethnicity  comparison  yielded  very  different  preferences 
for  American  Indians  and  Anglos.  American  Indians  chose  the  same  ethnicity 
(American  Indian)  characters  in  81%  of  their  pairings  with  Anglo  characters.  In 
contrast,  Anglos  chose  the  same  ethnicity  (Anglo)  characters  in  only  25%  of  their 
pairings  with  American  Indian  characters.  This  difference  in  preference  on  the  same- 
opposite  dimension  was  highly  significant,  %2  (h  160)  =  20.36,  £  <  .0001 .  Boys'  and 
girls'  ethnicity  preferences  did  not  differ  significantly  from  one  another. 
Gender  of  Main  Character 

Table  2  presents  the  proportion  of  subjects  who  chose  characters  of  the  same 
gender,  characters  of  both  genders,  and  characters  of  the  opposite  gender  in  three 
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pairings  of  illustrations.  Both  American  Indian  and  Anglo  children  indicated  a  strong 
preference  for  the  same  gender  illustrations  over  those  with  the  opposite  gender. 
American  Indians  selected  the  same  gender  over  the  opposite  in  97%  of  the  pairings. 
Anglos  chose  the  same  gender  over  the  opposite  in  95%  of  these  pairings.  American 
Indian  and  Anglo  subjects  chose  illustrations  showing  both  genders  over  those  showing 
either  the  opposite  gender  or  the  same  gender.  American  Indians  selected  both  genders 
over  the  opposite  in  66%  of  the  pairings  and  both  over  the  same  in  63%  of  the  pairings. 
Anglos  chose  both  over  the  op-  >site  in  75%  of  the  pairings  and  both  over  the  same  in 
60%.  Overall,  illustrations  depicting  the  subjects  same  gender  were  chosen  67%  of  the 
times  when  paired  with  the  other  two  choices,  both  genders  were  selected  in  66%  of  the 
pairings,  and  the  opposite-gender  in  17%  of  its  pairings. 

The  comparison  between  both  genders  vs.  the  opposite  gender  yielded  very 
different  preferences  for  the  boy  and  girl  subjects.  The  boys  selected  both  genders  in 
94%  of  the  pairings  with  the  opposite  gender.  In  contrast,  the  girl  subjects  selected  both 
genders  in  only  46%  of  the  pairings  with  the  opposite  gender.  There  was  a  highly 
significant  difference  in  this  preference  between  both  genders  or  the  opposite  gender,  y} 
(1,  160)  =  42.976,  p  <  .0001.  Similarly,  64%  of  the  boys  selected  the  illustration 
depicting  boys  over  the  one  with  both  genders,  but  only  14%  of  the  girls  selected  girls 
over  both  genders,  (%2  (1,  N  =  160)  =  42.133,  £  <  .0001).  There  were  no  significant 
differences  between  the  gender  preferences  of  the  American  Indian  and  Anglo  subjects. 
Story  Setting 

Table  3  shows  the  mean  number  of  suburban  illustrations  chosen  by  ethnicity 
and  gender.  The  table  reveals  that  the  160  students  selected  an  overall  average  of 
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2.46  suburban  settings  and  only  .54  reservation-like  settings  from  three  possible  choices. 
Anglo  children  chose  an  average  of  2.56  suburban  settings  and  Indian  children  chose  2.35 
suburban  settings.  Boys  selected  2.49  suburban  settings  and  females  selected  2.43. 
Ninety-four  percent  (N  =  150)  of  the  subjects  chose  all  three  of  the  suburban  settings 
(54%)  or  two  of  three  suburban  settings  (40%).  Only  three-  subjects  (2%)  selected  all 
three  reservation-like  illustrations. 

A  2  (ethnicity)  x  2  (gender)  analysis  of  variance  (ANOVA)  of  the  data  in  Tabic 
3  revealed  that  the  number  of  suburban  settings  chosen  by  Anglos  (m  =  2.56)  was 
significantly  higher  that  the  number  chosen  by  American  Indians  (m  =  2.35),  Fid,  159) 
=  4.049,  p  <  0.05.  The  difference  between  boys  and  girls  was  not  statistically 
significant,  nor  was  the  ethnicity  by  gender  interaction. 
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Discussion 

Two  characters  were  depicted  in  each  of  the  illustrations  for  ethnicity  and 
gender  of  main  character.  The  reason  for  using  two  characters  instead  of  one  is  that 
much  of  the  research  on  children's  preferences  make  the  child  select  between  one 
variable  or  the  other.  The  weakness  in  this  method  is  that  it  forces  a  choice  between 
variables  when  in  fact  both  can  be  incorporated  into  a  story.  This  study  was  designed  so 
the  children  can  make  the  choice  of  reading  about  both  ethnicities  or  both  genders. 

The  findings  from  this  study  have  positive  implications  for  fostering  the 
concept  of  ethnic  diversity  in  children.  American  Indian  and  Anglo  subjects  consistently 
preferred  to  read  about  characters  from  both  ethnic  groups  over  those  from  their  own 
group  or  from  the  opposite  group.  Interestingly,  Anglo  subjects  also  showed  a  strong 
preference  for  reading  about  the  opposite  ethnic  group  (American  Indians)  over  their 
own  ethnicity. 

The  results  from  three  studies  investigating  friendship  choices  of  primary 
grade  children  suggest  that  children  prefer  friends  of  their  own  ethnicity  (Braha  & 
Rutter,  1980;  Davey  &  Mullin,  1982;  Denscombe,  1983).  However,  this  trend  for  same 
ethnicity  friends  did  not  transfer  to  the  subjects  preferences  for  which  ethnicity  they 
would  most  like  to  read  about.  The  subjects  in  this  study  preferred  to  read  about  both 
ethnicities  over  their  same  ethnicity. 

A  very  striking  difference  is  present  in  the  subject's  responses  to  reading  about 
characters  of  their  same  ethnicity  over  the  opposite  ethnicity.  Eighty-one  percent  of 
the  American  Indian  children  preferred  to  read  stories  with  American  Indian  main 
characters.  Only  25%  of  the  Anglo  subjects  chose  to  read  stories  with  Anglo  children. 
Perhaps  the  Indian  children  indicated  a  preference  to  read  about  their  own  ethnicity 
because  they  seldom  get  to  read  about  themselves  and  would  like  to. 

Anglo  children,  on  the  other  hand,  nearly  always  have  the  opportunity  to  read 
about  their  own  ethnic  group.  Their  preference  for  stories  with  American  Indian 
children  as  the  main  characters  would  appear  to  reflect  a  desire  for  greater  variety. 
They  may  feel  that  stories  about  Indian  children  have  the  potential  to  be  more 
interesting  than  stories  about  children  like  themselves. 

Ninety-six  percent  of  both  male  subjects  and  female  subjects  preferred  stories 
with  main  characters  of  their  same  gender  rather  than  those  with  two  characters  of 
the  opposite  sex.  This  strong  preference  for  same-gender  over  opposite  gender 
characters  is  consistent  with  previous  research  on  children's  preference  for  gender  of 
main  character  (Harper-Marinick,  et  al.,  1990;  Beyard-Tyler  and  Sullivan,  1972;  Rose, 
Zimet  &  Blom,  1972). 

Boys  and  girls  differed  significantly  when  choosing  between  reading  about  both 
a  male  and  female  or  about  two  characters  of  the  opposite  sex.  The  majority  (94%)  of 
the  boys  preferred  to  read  about  both  a  boy  and  a  girl  instead  of  reading  about  two  girls. 
In  contrast,  less  than  half  (46%)  of  the  female  subjects  indicated  that  they  would  prefer 
to  read  about  both  genders  instead  of  two  boys.  This  difference  could  possibly  be 
attributed  to  boys  not  being  interested  in  reading  about  girls  by  themselves.  The  girls, 
however,  might  think  that  reading  a  story  with  two  boys  would  be  more  adventuresome 
than  a  story  with  both  a  boy  and  a  girl. 

Both  the  American  Indian  and  Anglo  subjects  indicated  a  strong  preference  for 
stories  with  a  suburban-type  setting  over  those  with  a  reservation -like  setting. 
Although  Anglo  subjects  had  a  significantly  stronger  preference  for  suburban  settings 
than  Indian  subjects,  this  pattern  was  not  a  strong  one  when  compared  to  the  consistent 
preference  for  the  suburban  settings  among  both  ethnic  groups.  Previous  studies  by 
Harper-Marinick,  et  al.  (1990)  and  Okada  and  Sullivan  (1971)  have  found  that  Black, 
Anglo,  and  Hispanic  inner-city  children  have  a  strong  preference  for  reading  about 
suburban  settings  over  the  inner-city  settings  in  which  they  live. 
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Suburban-type  settings  reflect  not  only  a  difference  in  location  from  the 
reservation-like  settings,  but  also  in  economic  level  because  the  setting  and  economic 
level  go  hand-in-hand.  Therefore,  children  may  make  their  selection  on  the  basis  of 
the  appeal  of  a  "nicer"  setting  rather  than  the  location  of  the  setting,  per  se,  even 
though  the  oral  directions  to  them  on  each  item  explicitly  note  the  setting. 

The  results  have  several  implications  for  authors  of  children's  reading 
material.  They  support  writing  stories  that  encourage  the  concept  of  ethnic  diversity 
by  having  main  characters  from  different  ethnic  backgrounds,  rather  than  having  a 
very  high  proportion  of  Anglo  main  characters.  This  would  include  a  selection  of 
stories  with  American  Indian  children  as  leading  characters.  Lankford  and  Riley 
(1986)  note  that  American  Indian  children  need  "culturally  relevant"  instructional 
material  infused  into  their  educational  programs.  The  present  results  indicate  that 
Anglo  children  may  also  benefit  from  the  availability  of  reading  material  that 
includes  American  Indian  characters. 

The  findings  suggest  authors  should  consider  providing  educational  material 
that  allows  children  to  read  stories  with  characters  of  their  same  gender,  and  also 
stories  that  have  both  a  boy  and  a  girl  as  main  characters.  These  findings  on  children's 
preferences  for  ethnicity  and  gender  of  main  character  fit  into  the  current  trend  of 
encouraging  multiculturalism  and  non-sexists  behavior  in  today's  educational  setting. 

Research  has  shown  that  a  student's  intrinsic  motivation  and  literal 
comprehension  increases  with  the  use  of  interesting  reading  material.  This  study  has 
revealed  that  children  are  interested  in  stories  with  more  than  one  ethnic  background 
represented  and  both  genders  represented.  It  has  also  shown  that  children  prefer  to 
read  stories  with  suburban-like  settings.  It  would  be  useful  to  conduct  further 
investigations  to  determine  if  children  read  more  frequently,  or  show  increases  in 
learning  when  provided  with  reading  material  which  reflects  individual  reading 
preferences.  Further  research  which  considers  reading  preferences  for  other  individual 
content  variables  and  various  combinations  of  variables  may  also  be  useful.  For 
instance,  would  an  Anglo  boy  who  wanted  to  read  about  two  American  Indian  boys  want 
to  read  about  two  American  Indian  girls?  Would  the  American  Indian  children  want  to 
read  about  a  suburban  school  with  two  Angio  boys?  Continued  research  in  children's 
preferences  could  prove  beneficial  to  future  reading  improvement  efforts. 
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Table  1 

Preferences  for  Ethnicity  of  Character? 


aPreferred  Ethnicity  and 
Preference  Level 


^Ethnicity 
of  Subjects 

Pairing  of 
Illustrations 

Boy 

Subjects 

Girls 
Subjects 

Totals 

American 
Indian 

Camp  i/C  fVrra (r> 

Same  vs.  Both 
Both  vs.  CftXHte 

Same 
Both 
Both 

.85 
.70 
.55 

Same 
Both 
Tie 

.77 
.70 
.50 

Same 
Both 
Both 

.81 
.70 
.53 

Anglo 

Same  vs.  Oj^poate 
Same  vs.  Btoth 
Both  vs.  Ojpposte 

Opp. 
Both 
Both 

,68 
.63 
.70 

Opp. 
Both 
Both 

.83 
.68 
.63 

Opp. 
Both 
Both 

.75 
.65 
.66 

Totals 

Same  vs.  O^poate 
Same  vs.  Both 
Both  vs.  Ofpoate 

Same 
Both 
Both 

.59 
.66 
.63 

Opp. 
Both 
Both 

.53 
.69 
.56 

Same 
Both 
Both 

.53 
.68 
.59 

These  columns  show  the  preferred  ethnic  group  for  each  pair  of  illustrations  and  the 
proportion  of  subjects  choosing  that  group.  For  example,  American  Indian  boys  chose 
the  illustration  depicting  their  same  ethnicity  (American  Indian)  over  the  one 
depicting  the  opposite  ethnicity  (Anglo)  85%  of  the  time,  both  ethnicities  (Indian- 
Anglo)  the  same  (Indian)  70%  of  the  time,  and  the  illustration  depicting  both 
(Indian-Anglo)  over  the  opposite(Anglo)  55%  of  the  time, 

*D=  80  (40  boys  and  40  girls)  per  ethnic  group. 
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Table  2 

Preferences  for  Gender  of  Characters 


aPreferred  Gender  and 
Preference  Level 


bGender 
of  Subjects 


Pairing  of 
Illustrations 


American 
Indian 


Anglo 


Totals 


Boy 
Subjects 


Boys  vs.  Qris 
Boys  vs.  Both 
Both  vs.  Qris 


Boys  .95 
Boys  .62 
Both  .92 


Boys  .98 
Boys  .65 
Both  .95 


Boys  .96 
Boys  .64 
Both  .94 


Girl 
Subjects 


Boys  vs.  Qris 
Boys  vs.  Both 
Both  vs.  Qris 


Girls  1 .00  Girls  .93  Girls  .96 
Boys  .60  Both  .53  Boys  .54 
Both     .88      Both    .85       Both  .86 


Totals 


Same  vs.  Opp. 
Both  vs.  Same 
Both  vs.  Opp. 


Same  .97 
Both  .63 
Both  .66 


Same  .95 
Both  .60 
Both  .74 


Same  .96 
Both  .61 
Both  .70 


aThese  columns  show  the  preferred  gender  for  each  pair  of  illustrations  and  the 
proportion  of  subjects  choosing  that  gender.  For  example,  American  Indian  boys  chose 
the  illustration  depicting  their  same  gender  (boys)  over  the  one  depicting  the 
opposite  gender  (girls)  95%  of  the  time,  the  illustration  depicting  boys  over  the  one 
depicting  both  a  boy  and  a  girl  62%  of  the  time,  and  the  illustration  depicting  both  (a 
boy  and  a  girl)  over  the  one  showing  the  opposite  (girl)  92%  of  the  time, 

bn=  80  (40  boys  and  40  girls)  per  ethnic  group. 
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Table  3 


Mean  Suburban  Settings  Selected  bv  Ethny  rirolip  and  Condor 
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Gender 


Ethnic  Groups3 


Anglo 

American  Indian 
Totals 


Boys 

Girls 

Totals 

2.63 

2.50 

2.56 

2.35 

2.35 

2.35 

2.49 

2.43 

2.46 

1  n  =  80  (40  boys  and  40  girls)  per  ethnic  group. 


666  Is* 


Reading  Preferences 
13 

Figure  Caption 

Figure  1.  Sample  illustrations  for  the  three  variables:  Ethnicity  of  main  character, 
Gender  of  main  character,  and  Story  setting. 
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Ethnicity  of  Main  Character  (male  subjects) 


An  American  Indian  and  an  An^Io  Two  Anglos 


Gender  of  Main  Character  (Anglo  subjects) 


Settings 


Suburban-like  home 


ERLC 


Reservation-like  home 
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ABSTRACT 

This  study  investigated  whether  the  visual  forms  of  buttons  influenced  learning  and  satisfaction 
when  early  adolescent  students  were  asked  to  perform  ts  using  computer-assisted  instruction 
(L  AI).  Ninety  subjects  participated.  Three  visual  forms  were  designed  for  buttons  located  in  a 
CAI  apphcauon.  One  contained  pictures  for  commands,  the  second  contained  words  for 
commands,  and  the  third  contained  the  combination  of  pictures  and  words.  Each  subject  was 
assigned  to  one  of  six  groups.  Each  group  used  two  of  the  three  styles  of  buttons.  The  groups 
performed  two  sets  of  three  tasks.  Response  time,  accuracy,  and  satisfaction  were  measured. 

The  results  indicated  that  groups  performed  faster  with  the  second  task  set.  The  results  also 
indicated  that  groups  were  more  accurate  in  their  responses  when  the  buttons  illustrated  words  than 
when  the  buttons  illustrated  the  combination.  Results  for  satisfaction  indicated  that  groups  were 
more  satisfied  with  the  combination  than  they  were  with  the  words. 


INTRODUCTION 

Students  who  use  CAI  will  not  tolerate  a  poorly  designed  interface  that  is  not  intuitive  at  first 
sight.  Smce  students  infrequendy  use  the  same  CAI  application  they  are  less  motivated  to  learn 
its  use.  Shncidcrman  ( 1987)  stated  that  if  the  students  do  not  understand  the  CAI  application  at 
El?K  th™a±f  wil1  J"?'*  aband°"  **  •»  of  the  computer  or  try  a  competing  package 
I  Ience.  the  CAI  application  s  interface  must  meet  their  needs.  Learning  users  should  be  able  to 
point  and  click  with  the  mouse  at  commands  illustrated  on  the  computer  screen.  Sec-and-ooint 
interfaces  usually  contain  buttons  illustrated  as  pictures,. as  words,  or  as  the  combination  of 
pictures  and  words  (Apple  Computer.  Inc.  1987).  For  adult  learners  of  computer  applications  it 
lus  been  found  that  performance  and  satisfaction  are  affected  by  the  visual  form  in  which  the 
commands  are  illustrated.  In  fact,  studies  have  found  that  pictures  (Temple.  Barker.  &  Sloane. 
1  m,  and  the  combination  of  pictures  and  words  (Egido  &  Patterson.  1988)  are  preferred  to  words 
by  adult  learning  users. 

Research  an  cognitive  development  seems  to  support  the  notion  that  pictures  and  the  combinations 
are  preferable  for  adults  when  adults  are  required  to  understand  them  as  representations  of 
information.  According  to  Bruner  et  ai.  (1966),  die  development  of  the  representations  of 
information  in  the  mind  is  identifiable  during  the  early  stages  of  life.  By  adolescence  through 
adulthood  the  number  of  mental  representations  increases  depending  on  experience  and  maturity  and 
may  include  visual  images  and  words  that  denote  a  single  idea.  Viewing  a  picture  or  the 
combination  of  a  picture  and  a  word  helps  adults  understand  and  discriminate  ideas  that  are 
represented  by  them. 

One  of  the  questions  Unit  arises  is  if  early  adolescents  will  perform  as  well  as  and  be  as  satisfied  ;ls 
adults  when  using  CAI  that  includes  pictures  and  the  combination  of  pictures  and  words  for 
buttons.  Do  early  adolescents  have  the  aptitude  to  understand  picture  labels  as  well  as  word  labels' 
Finally  does  the  combinauon  of  pictures  and  words  used  to  illustrate  commands  on  the  screen  tax' 
die  early  adolescent  s  memory  load?  In  order  to  answer  these  quesUons  the  following  study  was 
conducted.  1  he  purpose  of  the  study  was  to  determine  the  effect  of  the  three  styles  of  bullous 
pictures,  words  or  die  combination  of  pictures  and  words)  on  case  of  learning  and  computer 
interaction  satisfaction  when  early  adolescent  school  students  were  required  lo  use  a  CAI 
application.  ^  vm 
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METHOD 

Ninety  (44  males  and  46  females)  early  adolescent  school  students  participated  in  the  study.  Each 
student  was  randomly  assigned  to  a  group.  The  groups  saw  two  of  the  three  styles  of  buttons: 
picture,  words,  or  the  combination  of  pictures  and  words.  Group  A  used  pictures  first  and  words 
second,  while  group  B  used  words  first  and  pictures  second.  Group  C  used  the  combination  of 
pictures  and  words  first  and  words  second,  while  group  D  used  pictures  first  and  the  combination  of 
pictures  and  words  second.  Group  E  used  words  first  and  the  combination  of  pictures  and  words 
second,  while  group  F  used  the  combination  of  pictures  and  words  first  and  words  second.  The 
tests  were  administered  during  and  after  school.  The  study  took  approximately  forty  to  fifty 
minutes  to  complete. 

'Hie  CAI  application  contained  a  collection  of  facts  on  history.  The  students  were  able  to  search 
through  subject  areas  contained  in  the  application  easily  by  pointing  and  clicking  with  a  mouse  on 
the  buttons  located  at  the  bottom  of  the  computer  screens.  The  picture  labels  for  the  buttons  were 
black  and  white  and  the  words  were  displayed  in  Times  Roman  at  10  points.  The  buttons  were 
approximately  1/2"  sqwirc. 

The  CAI  application  was  a  commercially  available  package  that  was  modified  for  experimentation. 
It  originally  had  picture  labels  for  the  buttons.  These  labels  were  revised  to  words  and  a 
combination  of  pictures  find  words  based  on  the  description  of  the  function  of  the  buttons  found  in 
a  Help  section  of  the  CAI  application. 

liach  group  performed  a  set  of  tasks  (i.e.,  a  set  of  three  tasks)  while  using  one  of  the  two  styles  of 
buttons  assigned.  One  task  at  a  time  appeared  on  the  computer  screen  when  the  subjects  in  the 
groups  selected  a  button  labeled  Show  Task.  After  completing  the  set  of  tasks,  the  computer 
prompted  the  subjects  to  complete  a  satisfaction  questionnaire.  After  completing  the  questionnaire 
the  subjects  viewed  the  second  set  of  buttons  and  performed  the  second  set  of  tasks.  After 
completing  the  second  set  of  tasks,  satisfaction  was  assessed  again. 

Each  of  the  tasks  required  the  same  amount  of  mouse  motion  on  the  screen  and  required  the  same 
amount  of  branching  through  the  application.  The  computer  recorded  the  subject's  ID  number,  the 
buttons  selected,  think  time  which  was  the  elapsed  time  between  button  selections,  task  time 
which  was  the  total  elapsed  time  on  task,  and  total  task  time  which  was  the  total  elapsed  time  on 
the  set  of  tasks.  The  computer  process  time,  which  was  the  computer's  time  to  perform  the 
operation  required  when  a  button  was  selected  was  recorded  independently. 

The  button  selections  were  identified  and  a  rating  of  response  accuracy  was  recorded  for  each  set  of 
tasks.  The  scores  could  range  from  0  to  3  (0  =  no  correct  responses,  3  ■  all  correct  responses). 

The  students  had  access  to  each  task  while  searching  for  the  answer.  The  tasks  were  displayed  on 
the  computer  screen  when  the  button  labeled  Show  Task  was  selected.  The  time  spent  reading  the 
task  was  not  included  in  the  subjects1  performance  scores. 

A  user  interaction  satisfaction  questionnaire  was  used  to  measure  satisfaction.  Positive  satisfaction 
was  indicated  by  high  scores.  The  questionnaire  contained  13  items  with  9  point  scales.  Possible 
scores  ranged  from  0  to  1 17. 

DESIGN 

The  dependent  variables  response  tune,  response  accuracy,  and  satisfaction  were  recorded  and 
analyzed  in  three  two-by-two  Latin  squares  with  repeated  measures  designs.  The  Latin  square  with 
repeated  measures  design  was  selected  because  it  allowed  the  researcher  to  economize  on  subject 
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numbers  and  it  allowed  the  researcher  to  collect  data  on  a  number  of  variables  while  exposing  than 
to  more  than  one  treatment.  Sec  Figure  1  for  an  illustration  of  the  desian. 


Group 
A 
D 

C 
D 

r- 
p 


Task  Sgl  ' 
XiOj0203 
X2O1O2O3 

X3O1O2O3 
X1O1O2O3 

X2O1O2O3 
X3O1O2O3 


Task  Set  1 
X2O1O2O3 
XlO!0203 

X1O1O2O3 
X3O1O2O3 

X3O1O2O3 
X2O1O2O3 


Measurements  Treatments 


Conditions 
Task  Set  1 


0 1  =  Tune  X 1  =  Picture  Interface  Type  ,  ^  oei  , 

02  =  Number  correct  X2  =  Word  Interface  Type  Task  Set  2 
°3  -  Satisfaction                 X3  =  Picture  and  Word  Interface  Type 
Figure  !■  Diagram  of  experimental  design 


RESULTS 
DESCRIPTION  OF  THE  STUDY 


The  study  was  conducted  over  the  course  of  a  month  and  a  half.  Initially,  the  researcher  tried 
conducting  die  study  after  school  so  that  there  would  not  be  a  conflict  with  the  other  studies  in 
which  the  subjects  might  be  involved.  However,  it  was  found  after  several  days  of  testing  dial 
many  subjects  were  not  willing  to  participate  after  school.  Some  students  had I  other  activities  to 
during  SOL™5      'RtCreSt'  DUC  t0  P°°r  a,tendance'  ^  «sted  the  remaining  subjects 

Testing  the  subjects  during  school  resulted  in  other  problems.  The  computer  laboratory  at  the 
^rf!1fC1Ue!It,y,USedby  classcs>  Therese^^  found  that  there  was  litUe  time  between 
classes  to  collect  the  data  from  the  computers  before  another  class  wanted  to  use  them.  In  addition 

t3l  H  TLWhe"  lta  C°mpUter  hbotzioiy  was  not  •Bta*«"  for  teachersStudems 
came  to  the  laboratory  to  work  mdepcndcnUy  on  assigned  tasks. 

During  the  testing  periods,  the  procedures  were  explained  to  the  subjects  and  the  CAI  application 

Z^T,TC±  Whe"  *he  SubjCCtS  wcre  ,0,d  10  start-  ,na"y         contoSJi  to  2  to  do 
Some  asked  their  peers  what  they  were  to  do.  The  researcher  helped  many  subjects  get  sutted 

0£!?tV*  ^  U,C  SUbjeCtS  appCarCd  10  naVC  li,tle  »'  undiancfi  St was 

expected  of  diem.  The  researcher  accounted  for  meseiniUal  delays  during  the  tbua  analyses 
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ANALYSES 

I      square  with  repeated  measures  design  analyses  were  perfonned  to  determine  whether  there 
were  significant  mean  differences  in  the  groups,  tasks,  and  interfaces  on  the  dependent  variables: 
response  time,  response  accuracy,  and  satisfaction.  Hie  groups  and  tasks  were  evaluated  as 
independent  variables, 

Rg&mmra  Time 

The  results  from  the  analysis  of  each  independent  variable  for  response  time  indicated  that  there 
were  significant  differences  in  the  two  sets  of  tasks.  All  groups  performed  significantly  faster 
when  using  the  second  set  of  tasks.  Table  1  and  Table  2  display  the  results. 


Table  1. 


AIKIIVSIS  01  rcnonmnn^  nv  ias»  innmniiiM:  unit; 

Source 

If 

MS 

Emu 

E 

D 

Grps  A  &  B 
(Task  Unil) 

171396421.350 

1 

171396421.350 

Error 

5.491 

<.05 

Grps  C  &  D 
(Task  Unit) 

369614476.017 

1 

369614476.017 

Error 

14.649 

<.05 

Grps  li  &  F 
(Task  Unil) 

247786146.017 

1 

247786146.017 

Error 

9.701 

<.05 

Note:  Response  time  was  measured  in  ticks.  There  are  approximately  60.15  ticks  per  second  in 
I  lyperCard  (Apple  Computer,  1989). 


Table  2. 

Analysis  of  Menu  Scores  by  Task  for  Response  Time 


Task  Unit  1 

Task  Unit  2 

Source 

M  1 

SD 

M  1 

SB 

Grps  A  &  B 
Grps  C  &  D 
Grps  E  &  F 

13175.73 
13508.67 
12235.60 

7921.23 
7230.07 
8382.43 

9795.43 
8544.70 
8171.23 

6002.28 
4056.48 
4573.16 

Note:  The  number  of  subjects  in  each  group  =  15.  Response  time  was  measured  in  ticks. 
Response  Accuracy 

There  wjls  a  significant  mean  difference  in  response  accuracy  for  two  of  the  styles  of  buttons 
between  groups  E  (word  /  picture  &  word)  and  F  (picture  &  won!  /  word).  Table  3  and  Table  4 
display  the  results  from  the  analyses. 
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Table  3. 

Analysis  of  Performance  hv  Interfax  fnr  A.n|nirv  nf  o^mv 

<£S£?  017  1  0,7  «P  ,s. 

G(S^  3267  *•       "34  <.05 


Noie:  Overall  ratings  for  accuracy  of  response  could  range  from  0  to  3  (0  =  no  correct  responses  3 
=  all  correct  responses).  ' 

Table  4. 

Analysis  nf  Mean  Scares  hv  miMf**.  r™-  ftfriracv  nr  ppcppn^ 

S—  I   Pictures   1  Words  I       Pictures  &  Word* 

Source  |       M        I      3D       I      M       I      SD  \       M       I  SD  


Grps  1.30  .85  1.27  83 

A&D 

Grps  1.47  .78 
C&D 


1.43  .68 


GTi  ~  ~  1.50  .90  1.03  .62 


E&F 


^^m7^°r!Sm^in^Ch8mUP=  15'  0vera»  stings  for  accuracy  of  response  could 
range  from  0  to  3  (0  =  no  correct  responses,  3  =  all  correct  responses). 

Satisfaction 

There  was  a  significant  mean  difference  in  satisfaction  for  two  of  the  visual  forms  between  crouns 

toZ&Eg? word)  31,(1  F  (picture  &  word  1  wonl)- Tab,e  5 «'  Taw< 6 
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Table  5. 

Analysis  of  Performance  by  Interface  for  Interaction  Satisfaction 


Source  SS  df  MS.  Error     F(dH  p 


Grps  A&B 
(Interface) 

1.067 

1 

1.067 

Error 

.015 

n.s. 

Grps  C  &  D 
(Interface) 

1.667 

1 

1.667 

Error 

.061 

n.s. 

Grps  1-  &  F 
(Interface) 

576.60 

1 

576.60 

Error 

4.710 

<.05 

Note:  Computer  interaction  satisfaction  scores  could  range  from  0  to  117.  Positive  computer 
interaction  was  indicated  by  high  scores. 


Table  6. 

Analysis  of  Mean  Scores  by  Interface  for  Interaction  Satisfaction 


 |  Pictures  |  Words  I      Pictures  &  Words 

Source        M  SH  M  SD  M  SD 


Grps 
A&B 

75.83 

16.03 

76.10 

15.03 

Grps 
C&D 

81.30 

15.55 

81.63 

15.92 

Grps 
E&F 

76.53 

17.39 

82.73 

15.77 

Note:  The  number  of  subjects  in  each  group  =15.  Computer  interaction  satisfact*  ,i  scores  could 
range  from  0  to  1 17.  Positive  computer  interaction  was  indicated  by  high  scores. 


DISCUSSION 

'Hie  styles  of  buttons  used  for  the  commands  in  sce-and-point  interfaces  can  influence  a  learner's 
ability  to  use  CM.  Ilic  goal  is  to  select  the  appropriate  type.  The  learner  dealing  with  new 
information  should  be  able  to  easily  navigate  through  a  CAI  lesson.  Understanding  the  most 
appropriate  way  to  display  the  commands  becomes  increasing  important  as  students  are  required  to 
use  technology  to  leant.  Some  might  argue  that  the  type  of  button  used  for  a  command  depends 
on  what  concept  is  being  represented.  Sometimes  it  might  be  easier  to  use  a  word  to  illustrate  a 
command,  in  other  cases  it  might  be  easier  to  use  a  picture.  However,  when  faced  with  designing 
the  interface  it  is  important  to  understand  what  type  of  button,  in  general,  works  best  for  a 
particular  group  of  learners.  Thus,  the  present  study  provides  some  tentative  conclusions  that  can 
help  guide  learner  interface  design. 

'Hie  results  from  the  present  study  indicated  that  the  early  adolescent  school  students  in  groups  E 
(word  /  picture  &  word)  and  F  (picture  &  word  /  word)  were  more  accurate  in  responding  to  the 
computer  interface  that  included  words  only  compared  to  the  interface  that  included  the  combination 
of  pictures  and  words.  Perhaps,  computer  interfaces  that  incorporate  words  are  easier  to  identify 
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because  words  tend  to  be  less  ambiguous  for  early  adolescent  school  students.  For  example  for  a 
concept  such  as  -print  an  image  to  the  printer"  the  word  firiju  might  be  more  explicit.  A  picture 
of  this  concept  might  be  difficult  to  represent  especially  in  a  small  space  on  the  computer  screen. 

Another  possibility  is  that  the  combination  of  pictures  and  wr  «Js  provided  too  much  information 
compared  to  words.  The  computer  screen  may  have  appeare  >  overwhelming  in  a  testing  situation. 
The  subjects  may  not  have  been  able  to  focus  on  each  button  in  order  to  infer  a  meaning  from  it 
According  to  Galito  (1989),  the  degree  to  which  a  user's  memory  is  being  used  to  understand  the ' 
information  can  have  a  significant  impact  on  their  comprehension  of  the  information  If  the 
amount  of  information  is  too  much  the  user's  performance  will  be  negatively  influenced. 

It  is  interesting  to  note  that  the  groups  who  used  the  words  and  the  combination  of  pictures  and 
words  were  more  satisfied  with  the  combination  even  though  they  didn't  perform  as  accurately  with 
the  combination.  Tins  finding  illustrates  that  user  satisfaction  is  not  necessarily  a  good  predictor 
of  user  performance  According  to  Long.  Hammond,  Barnard.  Morton.  Clark.  (1983).  the  opinions 
of  usere  do  not  predict  how  well  users  wtil  perform.  *"i»m«*k> 

All  groups  were  faster  the  second  time  with  the  second  set  of  tasks.  Possibly,  as  students 
developed  an  understanding  as  to  how  to  navigate  through  the  program  the  speed  with  which  thev 
performed  increased.  However,  the  accuracy  in  their  performance  was  not  affected.  It  is  possible 
that  the  subjects  felt  pressured  to  finish  the  second  set  of  tasks  quickly  but  not  necessarily  more 
accurately  because  other  subjects  had  finished.  The  researcher  instructed  subjects  who  were  finished 
to  quietly  leave  the  computer  laboratory  or  do  another  activity.  However,  it  was  difficult  to  keen 
the  students  quiet  after  finishing  the  tests.  The  other  subjects  who  were  still  working  were 
probably  distracted  by  this  commotion  and  felt  a  need  to  quickly  finish. 

An  important  issue  is  whether  the  results  obtained  in  the  present  study  can  be  generalized  The 
CA I  application  used  in  the  present  study  represented  a  general  layout  which  is  seen  in  other  CAI 
applications,  lhe  buttons  appeared  in  black  and  white  on  the  bottom  portion  of  die  screens  ' 
However,  the  results  might  be  specific  because  only  a  small  sampling  of  representations  for 
commands  were  evaluated. 

"Which  style  button  should  CAI  use?"  This  question  remains  unanswered.  However,  the  present 
study  provides  some  tentative  conclusions  to  guide  learner  interface  design.  The  study  found  ' 
differences  between  the  words  and  the  combination  of  pictures  and  words  when  early  adolescents 
were  required  to  perfonn  search  tasks  using  CAI.  In  addition,  it  was  found  that  satisfaction  with  an 
mterface  does  not  necessarily  indicate  how  well  an  early  adolescent  school  student  wtil  accurlely 
performed  search  bisks  when  using  CAI.  From  these  findings  it  appears  that  there  is  a  difference 
be  ween  words  and  the  combination  of  pictures  and  words.  However,  further  research 'needs  u  be 
dome  in  order  to  substantiate  the  significance  of  these  findings. 
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Assessing  Computer  Use  ,.ou  Instructional  TrNunlorm*Uoii 


Hie  Levels  of  Computer  Use  asseaament  (LCU)  was  developed  to  classify 
teachers*  use  of  computers  so  that  variations  of  use  could  be  studied.  It  was  derived 
j     of  Instructional  Transformation  (Rieber  &  Welliver,  1989;  Welliver 
1990).  The  development  of  the  LCU  was  stimulated  by  the  identification  of  the 
discrepancy  between  advocacy  for  the  use  of  computers  in  education  and  their  actual 
use  by  teachers. 

It  is  widely  believed  that  computer  technology  can  help  improve  the  educational 
system  (see  The  National  Task  Force  on  Educational  Technology,  1986  United 
States  Office i  of  Technology  Assessment  [OTA],  1988;  Sheingold  and  Hadley,  1990; 
Shanker  1990).  At  the  same  time,  the  availability  of  computers  in  schools  has 
increased.  In  the  period  from  1983  to  1987,  the  average  pupil-to-computer  ratio  in 
public  elementary  schools  improved  from  112.4  to  36.8  pupils  per  computer  (OTA) 

Despite  the  positive  trends  in  computer  availability,  relatively  few  teachers  have 
integrated  computers  into  teaching.  A  recent  national  survey  has  shown  that  even  in 
schools  where  computers  were  available  at  more  than  double  the  national  average 

,  o°Qnoe  P,!f  ^00i,had  ^g^d  *e  computer  into  the  classroom  ' 

(Becker,  1989;  Sheingold  &  Hadley,  1990).  Apparently,  sheer  availability  of 
computers  did  not  result  in  their  increased  integration  into  teaching. 

Indeed,  there  is  a  strong  sentiment  of  disappointment  and  disillusionment 
t^0"^8^™^  1987;  Bjerklie  &  Hollis, 

1991;Dronka,  1985;  Ely  &  Plomp,  1988;  O'Neill,  1990;  Roblyer,  1988).  The  poor 
opinion  of  educational  computing  by  teachers  does  not  agree  with  the  views  of  its 
supporters  and  patrons  in  government,  academe,  and  teachers'  unions  Thk 
discrepancy  in  perceptions  of  educational  computing  deserves  study  because  the 
resuit  of  it  is  that  computers  in  education  are  underutilized.  As  a  result  whatever 
hopes  there  are  for  educational  computing  to  affect  education  may  never  be  realized 
r  or  the  research  on  teachers*  computer  use,  a  fundamental  question  is  "What  is 
meant ;  by  use  ?'  This  question  has  been  addressed  in  the  model  of  Instructional 
lranstormation  suggested  by  Rieber  and  Welliver  (1989). 


Instructional  Transformation 


The  modelof  Instructional  Transformation  describes  the  process  of  the  adoption 
ol  innovation  This  is  an  area  of  human  behavior  that  has  been  studied  generally - 

°1Ja(ioPtl°n  behavior  based  on  length  of  time  have  been  described  (Rogers, 
1983  Rogers  &  Shoemaker,  1971 )  and  specifically-a  framework  for  analyzing  the 
adoption  of  innovations  among  educators  has  been  described  (Hall,  Loucks 
Rutherford  &  Newlove  1975).  The  model  of  Instructional  Transformation  focuses 
on  the  adoption  of  a  specific  educational  innovation  -  the  computer-and  describes  a 

W«lllvpr  W  w°,r  ^Sn^8^868  of  involvement  with  computers  (Rieber  and 
Welliver,  1989;  Welliver,  1990).  First,  a  teacher  becomes  familiar  with  computers 
(familtamation i );  then  the  teacher  uses  computers  in  teaching  {utilization).  A  higher 

evel  of  use  ia  observed  when  the  computers  have  become  critical  to  the  teaching 
(integration ).  An  analogy  to  this  scenario  would  be  of  one  who  upon  learning  to  drive 
[familtamation )  usea  the  car  to  improve  one'a  general  transportation  in  order  to 

2rJff°iPTu  utilization ).  One  may  then  schedule  the  use  of  the  car  so 
that  the  vehicle  ia  critical  to  the  function  of  fulfilling  the  purpoaea  ( intcgmtion )  In 
education,  this  atage  includea  the  teacher'a  awareneaa  of  tKiange  offua  oHicr  role 
With  this  new  awareness  of  the  restructuring  of  teaching  activitiea,  the  teacher 
JSSSJSi  °XPH,T°r  l",ld/lMe  ^P8  of  the  c°mputer-teacher-8tudent  relationship 
«S2wff?    T?f        8teiC  {evolu*wn)  iB  more  ^ a  auggeation  than  a  condition 
and  that  ia  to  continue  practicing  and  learning  about  how  to  develop  the 

tSEoSJ,l,t    inatrUctlon  thr0U8h  the  systematic  implementation  of  computer 

The  definition  of  use  for  the  Levels  of  Computer  Use  assessment  was  derived 
from  the  model  of  Instructional  Transformation.  Use  was  defined  hs  the  integrated 
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employment  of  computers  in  teaching.  Two  levels  of  use  were ^designated -Utilization 
and  Integration.  At  the  Utilization  level,  a  teacher  shares  and  delegates  teaching 
duties  to  the  computer;  however,  the  absence  of  the  computer  would  not  prevent  the 
implementation  of  instruction.  At  the  Integration  level,  teaching  duties  would  also  be 
shared  by  and  delegated  to  the  computer,  however,  the  absence  of  computers  would 
prevent  the  implementation  of  instruction.  The  distinction  between  the  Utilization 
and  Integration  levels  lies  in  the  expendability  of  the  computer  technology.  ^ 
Expandability  describes  the  relationship  of  computer  technology  to  a  teacher  s 
planned  instruction -whether  or  not  instruction  would  be  able  to  continue  in  the 
hypothetical  event  of,  say  the  sudden  absence  of  computer  technology  It  is  this 
dimension  of  expendability  that  was  identified  as  the  cut-off  for  membership  in  either 
category.  According  to  the  model  of  Instructional  Transformation,  the  Integration 
stage  is  further  characterized  by  the  dimension  of  a  teacher's  emergent  self- 
awareness  of  a  role  change  in  teaching  from  teacher-centered  to  learner-centered.  In 
a  departure  from  the  model,  the  assessment  does  not  encompass  the  dimension  of 
awareness  of  role  change. 


Development  of  the  Assessment 


The  LCU  is  the  result  of  several  attempts  at  varying  formats.  In  the  original 
conception  of  the  assessment,  items  were  written  for  each  of  the  five  stages  of  the 
model  of  Instructional  Transformation.  This  had  been  field  tested  and  revised  several 
times  and  was  finally  supplanted  by  the  current  format.  Hie  earlier  forms  of  the 
assessment  adhered  to  the  progressive  nature  of  the  model  and  attempted  to  capture 
all  of  its  dimensions,  however,  it  was  difficult  to  classify  responses.  This  is  because 
there  are  several  dimensions  that  run  throughout  the  model  in  a  progressive 
overlapping  manner.  In  the  current  form,  one  dimension  is  assessed.  The  most 
appropriate  technique  for  this  sort  of  assessment  is  the  paired  comparisons 
technique.  This  technique  requires  that  a  discrete  definition  be  operationalized  to 
determine  the  cut  off  for  the  levels  of  use.  Additionally,  the  levels  need  to  be  mutually 
exclusive  and  exhaustive.  Nunnally  (1959)  considers  this  method  "the  most  exact 
psychophysical  tool ... n  useful  for  precise  information  concerning  judgments  or 
preference.  ,  .  ... 

In  the  format  of  the  LCU,  each  item  from  one  level  is  paired  alternately  with 
each  item  from  the  other  level  (See  Table  1).  For  example,  if  there  were  four  items, 
two  from  the  first  level  (  la,  lb)  and  two  from  the  second  level  (2a,  2b),  the  paired 
comparisons  would  be  (la/2a),  (la/2b),  (lb/2a),  and  (lb/2b).  The  respondent  is  forced 
to  endorse  one  item  per  pair.  Hie  paired  comparisons  technique  allows  only  one 
combination  of  responses  to  indicate  membership  at  either  level.  There  are  four  pairs 
of  items  in  this  version.  Two  items  indicate  the  Utilization  level  and  two  the 
Integration  level  If,  for  example,  a  value  of  1  were  assigned  to  responses  indicating 
the  Utilization  level,  consistent  responses  for  4  items  would  score  4  points.  A  value  of 
2  for  responses  indicating  the  Integration  level,  would  result  in  consistent  response 
scoring  of  8.  Acceptable  scores  would  be  4  or  8.  Scores  of  6  or  7  indicate  an 
inconsistency.  The  four  items  were  selected  from  a  field  testing  with  a  sample  of  50 
elementary  school  teachers  because  those  items  had  the  highest  response 
consistency.  In  that  testing,  there  were  15  items.  Items  which  generated  response 
Inconsistency  or  which  introduced  an  additional  dimension  were  eliminated. 

In  a  subsequent  field  testing,  twenty-three  elementary  school  teachers 
responded;  there  were  2  inconsistent  responses.  Hie  estimated  reliability  of  the  LCU 
for  the  second  field  trial  using  the  Coefficient  of  Reproducibility  ( CR)  was  .74.  One  of 
the  items  was  reworded.  Hie  LCU  was  then  administered  to  170  elementary  school 
teachers.  The  CR  was  estimated  at  .96.  A  CR  of  .90  is  the  criterion  for  demonstrating 
that  items  form  an  ordered  scale  of  allowable  response  patterns  (Crocker  &  Alguia, 
1986).  The  teachers  were  also  asked  to  respond  to  a  control  item  about  their  self- 
reported  computer  use;  the  responses  were  matched  with  those  of  the  LCU  and 
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provided  additional  data  for  estimating  criterion-related  validity.  The  consistency  of 
classification  of  the  measure  was  estimated  computing  Cohen's  kappa  (Kappa  =  .72), 
The  results  are  important  for  two  reasons:  1)  the  estimated  reliability  is  high,  and  2) 
the  demonstration  of  an  ordered  scale  suggests  that  the  model  of  Instructional 
Transformation  identifies  at  least  two  progressive  levels. 


Table  1 

The  Levels  of  Computer  Use  assessment 


1.  *a.  In  my  instruction,  the  use  of  the  microcomputer  is  supplemental, 
•b.  The  microcomputer  is  critical  to  the  functioning  of  my  instruction. 

2.  *a.  The  use  of  the  microcomputer  is  not  essential  in  my  instruction, 
•b.  For  my  teaching,  the  use  of  the  microcomputer  is  indispensable. 

3.  »a.  The  microcomputer  is  critical  to  the  functioning  of  my  instruction. 
*b.  The  use  of  the  microcomputer  is  not  essential  in  my  instruction. 

4.  »a.  For  my  teaching,  the  use  of  the  microcomputer  is  indispensable. 
*b.  In  my  instruction,  the  use  of  the  microcomputer  is  supplemental. 

Note;     •  indicates  an  item  of  the  integration  level. 
*  indicates  an  item  of  the  utilization  level. 


The  LCU  has  been  used  in  several  studies.  It  has  been  used  to  establish  a 
dependent  variable -level  of  computer  use-in  a  study  of  elementary  school  teachers 
(Marcmfcewicz,  1991,  in  press)  and  in  a  longitudinal  study  of  preservice 
undergraduate  teachers  (Marcinltiewicz  &  Grabowski,  1991).  It  has  been  adapted  by 
the  Grosse  Pointe  (Michigan)  Public  School  System  for  use  in  its  Staff  Computer 
Skills  Survey  (J.  Corbett,  personal  commxinication,  February  21, 1992).  It  is  being 
used  to  establish  a  dependent  variable -levels  of  computer  use -relative  to  the 
subjective  norms  of  rural  teachers.  It  should  be  noted  that  prior  to  this  writing  the 
assessment  has  been  referred  to  as  the  LU  assessment.  As  of  t!iis  discussion  the 
assessment  is  referred  to  as  the  LCU-Levels  of  Computer  Use -assessment.  This 
modification  has  been  made  to  reflect  the  purpose  of  the  assessment  more  accurately 
This  modification  also  underscores  the  ongoing  study  of  the  assessment  and  of  the 
model  or  Instructional  Transformation. 


Discussion 


There  is  a  fundamental  notion  in  computer  implementation  that  is  often 
overlooked -not  all  intended  computer  users  do  use  them.  Aa  with  most  innovations 
people  vary  in  their  adoption  of  computer  use.  By  focusing  on  a  specific  profession 
such  as  teaching,  the  study  of  hov  its  members-teachers-differ  in  their  use  of 
computers  is  more  manageable.  Tney  differ  before  they  begin  to  use  computers  and 
they  differ  once  they  have  begun  to  use  them.  Before  use,  they  differ  in  the  time  it 
takeB  to  adopt  the  idea  of  using  the  technology.  Aftei  teachers  ad  ->t  the  notion  of 
using  computers,  they  differ  in  the  nature  of  their  computer  use.  The  importance  of 
the  model  of  Instructional  Transformation  is  that  it  addresses  the  fundamental 
variance  in  the  adoption  of  computer  use.  It  describes  the  progression  of  a  teacher 
irom  the  onset  of  the  adoption  of  computers.  In  application,  this  theoretical  model 
will  help  educators  in  several  ways:  1)  to  identify  the  range  of  expected  behaviors 
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applicable  to  a  teacher's  computer  use-what  can  we  expect  a  teacher  to  become?;  2) 
to  identify  the  level  of  a  teacher's  computer  use;  3)  to  identify  whether  a  teacher  is 
progressing  in  computer  use;  4)  to  identify  how  the  nature  of  a  teacher's  instruction 
changes  due  to  his  or  her  computer  use;  5)  to  identify  how  the  nature  of  teacher's 
professional  self-perception  changes  due  to  computer  use .  . 

When  educators  can  identify  these  aspects  of  teachers,  the  professional  growth 
and  development  of  teachers  relative  to  computer  use  can  be  understood.  With  this 
understanding  there  is  a  basis  for  recommending  staff  development,  remediation,  or 
differential  staffing.  Hie  LCU  assessment  was  developed  so  that  educators  could 
implement  the  model  of  Instructional  Transformation.  Some  applications  of  the 
assessment  include  assessing  a  school's  teachers  to  determine  the  current  computer 
utilizationAmderutilization  levels  before  a  purchase  order  for  computers  is  issued,  so 
that  the  likelihood  of  future  use  can  be  estimated.  Also,  as  a  school  is  designing  its 
restructuring  plans,  the  LCU  can  be  used  in  needs  assessment  to  identify 
discrepancies  in  the  levels  of  computer  use.  A  school  may  also  study  the  assessment 
and  the  model  of  Instructional  Transformation  to  learn  what  the  extent  of 
possibilities  of  computer  use  in  education  are.  In  doing  so,  a  school  can  be  helped  in 
clarifying  its  goals  in  the  application  of  computer  technology. 

The  LCU  accomplishes  the  classification  of  teachers  into  two  levels  of  computer 
use.  When  used  for  this  purpose,  the  assessment  addresses  point  #2  mentioned 
above.  The  current  form  of  the  LCU  is  markedly  different  from  earlier  versions.  The 
way  in  which  the  current  and  past  forms  differ  needs  to  be  pointed  out  to  highlight 
the  strength  of  the  current  version  as  well  as  to  identify  directions  for  future 
research.  Most  importantly,  the  past  and  current  versions  differ  in  that  the  former 
comprised  several  dimensions— as  does  the  model— attempting  to  measure  all 
dimensions.  Hiis  proved  to  be  confounding  since  the  dimensions  share 
characteristics.  It  was  decided  that  it  was  not  practical  or  perhaps  possible  to  capture 
all  the  dimensions  simultaneously.  In  the  LCU  assessment,  there  is  one  dimension - 
the  expendability  of  computers -that  is  measured.  This  is  the  central  dimension  of 
the  model.  Measures  of  the  other  dimensions  need  to  be  developed;  such  as  for  the 
degree  of  awareness  of  the  changing  of  one's  role  as  a  teacher. 

The  successful  development  of  the  LCU  establishes  it  as  a  part  of  the 
assessment  arm  of  the  model  of  Instructional  Transformation.  The  model  is  useful  in 
understanding  a  teacher's  theoretical  progression  in  the  adoption  of  computers.  Hie 
assessment  offers  a  means  for  applying  the  theoretical  model. 
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Abstract 

What  happens  when  a  globrJ  classroom,  using  computer  mediated  communication 
(CMC),  connects  students  internationally?  This  paper  reports  the  experiences  of  faculty 
and  students  at  a  Turkish  university,  Anadolu  University,  who  participated  In  Globaleil.  a 
global  education  computer  mediated  communication  project.  This  project,  initiated  at 
University  of  New  Mexico,  iiu .  Jded  faculty  and  students  at  University  of  Oklahoma, 
Florida  State  University,  Arizona  State  University,  University  of  Wyoming  and  Anadolu 
University  in  Turkey. 

The  paper  begins  by  examining  the  question  of  empowerment,  the  potential  of  ihe 
computer  to  offer  information  to  people  who  have  traditionally  been  information  poor,  by 
providing  them  with  networked  data  through  the  electronic  classroom  setting.  The 
discussion  proceeds  to  describe  how  computer-mediated  communication  can  address  this 
question  of  empowerment.  Following  the  discussion  is  a  case  summary  of  the  participation 
of  students  at  Anadolu  University,  Turkey,  in  a  pilot  project  witli  Glohalcd  during  the 
Spring  Semester  of  1 992, 

As  a  result  of  participation  in  the  Globuled  program,  certain  issues  emerged  whit  h 
shouid  be  taken  Into  consideration  if  future  global  classroom  situations  are  to  be 
successful.   Issues  of  1)  pedagogy.  L)  motivation.  3)  access  to  information,  4)  let  hnlcal 
difficulties  and  5)  language  need  to  be  addressed.  Finally  the  benefits  of  a  global 
classroom,  as  identified  by  participants,  are  discussed.  Enthusiasm  for  computer- 
mediated  communication  as  an  inexpensive  way  to  gain  access  to  information,  provide 
opportunities  for  practicing  a  foreign  language,  and  to  introduce  friends  across 
international  boundaries  were  the  most  frequently  cited  advantages.  In  conclusion, 
important  questions  must  be  answered  such  as  what  networked  information  will  be  made 
available,  for  whom  and  under  what  conditions,  in  order  for  global  education  to  trui> 
provide  equal  opportunity  for  ail  participants. 


Computer-mediated  communication  (CMC)  is  one  example  of  an 
interactive  technology  which  allows  the  user  to  have  access  to  a  wide  variety 
of  resources  using  existing  global  networks.  The  use  of  CMC  for  delivery  of 
instruction  in  industrialized  countries  has  been  well  documented  (Phillips, 
Santoro  &  Kuehn,  1989),  (Phelps,  Wells,  Ashworth  &  Hahn,  1991),  (Cheng, 
Lehman  &  Armstrong,  1991).  In  examining  the  issue  of  equity  and  computers, 
Sutton  (1991)  concluded  that  considerable  research  has  been  conducted  on 
equity  issues  among  white  middle-class  students,  but  that  much  remains  to  be 
done  in  examining  equity  issues  among  other  populations.  Davie  and  Wells 
(1991)  make  a  strong  case  for  the  empowering  attributes  of  CMC  in  the 
electronic  classroom  setting. 
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What  has  received  less  attention  is  the  degree  to  which  CMC  on  a  global 
scale  empowers  those  in  cross-cultural  or  cross-national  settings.    For  the 
first  time  users  in  such  settings  can  access  worldwide  information  sources, 
select  materials  appropriate  to  their  topics,  discuss  ideas  with  colleagues 
around  the  globe,  encounter  new  ideas,  and  make  significant  contributions  in 
a  truly  global  exchange  of  thought.  This  paper  explores  the  Implications  of 
such  global  information  exchange  using  CMC  in  settings  which  have 
traditionally  been  termed  "information  poor". 

Empowerment 

To  empower  is  to  give  power  to  those  persons  who  for  social,  economic 
or  political  reasons,  did  not  previously  have  that  power.  The  movement  to 
provide  empowering  opportunities  developed  as  a  result  of  the  struggle  to 
provide  people  with  equal  educational  opportunities,  so  that  they  could 
participate  and  make  well  informed  decisions  in  the  global  community.  In 
order  to  do  that,  access  to  information  quickly  became  the  key  factor.  At  the 
same  lime,  telecommunications  technology  developed  to  the  point  that 
information  databases  can  now  provide  answers  to  questions  within  minutes. 
Access  to  such  information  gives  the  user  a  sense  of  self-confidence  in  being 
able  to  make  well-informed  decisions. 

For  too  long,  empowering  tools  and  information  sources  have  been 
available  primarily  to  the  affluent  dominant  cultures  whose  decisions  have 
affected  not  only  themselves  but  the  world  at  large.  This  historical  trend  can 
be  traced  to  the  Middle  Ages  when  the  people  who  had  access  to  written 
materials  determined  the  distribution  of  information  which  was  interpreted 
and  later  made  available  to  the  masses.  The  Gutenberg  revolution  changed  the 
way  people  received  information  and  It  was  no  longer  necessary  to  have 
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information  interpreted  before  it  became  available  to  all.  With  the  most 
recent  technological  revolution,  and  the  development  of  computerized 
information  networks,  we  have  moved  one  step  closer  to  providing  truly  equal 
access  to  information  around  the  world.  The  availability  of  such  information 
allows  the  user  of  computer  networks  the  satisfaction  of  being  able  to 
communicate  with  others  on  a  global  scale,  sharing  information,  concerns  and 
points  of  view.  These  new  users  are  participating  as  colleagues  contributing 
their  own  resources,  not  simply  as  recipients  of  information,  and  they  are 
making  their  own  contributions  to  network  dialog.  When  people  become 
active  participants  in  decision  making,  their  curiosity  and  interest  in  the 
educational  process  and  in  self-improvement  begin  to  grow.  As  numerous 
research  studies  indicate,  when  given  the  opportunity  students  enjoy  using 
technologies  such  as  computers  and  networks  for  learning. 

Technologies  are  considered  by  many  to  be  empowering  tools.  The 
access  to  information  provided  by  computer  networks  is  believed  to  provide 
equal  opportunity  for  users  who  want  to  participate  in  decisions  of  the  greater 
global  community.  Many  feel  that  the  division  between  wealthy  and  poor 
countries,  that  is  the  gap  between  the  'haves'  and  the  'have-nots',  will  be 
lessened  by  the  acquisition  of  knowledge  through  networks.  These  networks 
can  give  individuals  a  new  confidence  in  their  ability  to  read  and  synthesize 
information  which  is  so  abundantly  available  in  Western  countries.  For  this 
reason,  many  developing  countries  like  Turkey  have  invested  heavily  in 
modern  technologies  of  instruction. 
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Computer-mediated  communication 

Computer-mediated  communication  (CMC)  is  an  electronic  system  using 
telecommunications  for  bulletin  boards,  electronic  mail,  and  computer 
conferencing.  It  is  computer  conferencing,  the  ability  to  use  the  computer 
for  group  work  independent  of  time  and  location,  which  promises  the  greatest 
opportunities  for  empowerment  (Waggoner,  1992).  Many  educators  have 
expressed  the  notion  that  CMC  has  the  potential  to  emerge  as  a  new  educational 
paradigm,  sharing  the  stage  with  traditional  education  and  various  forms  of 
distance  education,  but  unique  in  the  interactivity  it  provides  independent  of 
time  and  space  (Kaye,  1989).  This  interactivity  promotes  collaborative 
learning  in  a  less  restrictive  setting  than  has  been  thus  far  possible.  Now 
students  in  different  locations  can,  asynchronously  24  hours  a  day,  participate 
in  on-line  seminars,  on-line  discussion  groups  and  other  collaborative 
learning  activities  (Harasim,  1989). 

Global  classroom  networks  which  use  computer-mediated  systems 
encourage  international  dialog  and  resource  sharing.    One  of  the  pioneers  in 
developing  the  concept  of  the  global  classroom  is  Takeshi  Utsumi,  President  of 
GLOSAS  (Global  Systems  Analysis  and  Simulation)  who  has  worked  to  develop 
models  of  the  "Global  University"  and  the  "Global  Lecture  Hall"  which  provide 
resources  allowing  less  affluent  countries  to  keep  up  with  advances  in  global 
research  and  education  (Utsumi,  1990).  The  global  classroom,  using  the 
network  resources  of  international  research  networks  like  BITNET,  EARNET 
and  INTERNET,  can  provide  online  discussion  opportunities  for  students  around 
the  world.  One  such  global  classroom  was  the  Global  Education  Project 
(Globaled)  designed  by  Professor  Gunawardena  at  the  University  of  New 
Mexico  in  the  Spring  semester  of  1992.  Four  universities  participated  in  a 
global  computer-mediated  classroom  on  the  topic:  Theory  and  Practice  of 
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Distance  Education.  Three  of  the  universities  were  in  the  United  States  and  one 
was  in  Turkey. 

During  the  Spring  semester  of  1992,  students  from  Anadolu  University 
participated  in  this  Globaled  classroom,  a  classroom  without  walls  which  had  as 
its  focus  the  discussion  of  issues  related  to  educational  technology  and  distance 
education.  The  participating  class  at  each  university  submitted  a  question 
related  to  distance  education  which  other  students  were  encouraged  to  answer, 
debate,  and  question.  In  addition,  students  were  encouraged  to  respond  directly 
to  others  and  form  subgroups  if  their  topics  proved  particularly  interesting. 
One  goal  of  the  exchange  was  to  develop  in  students  the  ability  to  analyze  and 
synthesize  information  as  well  as  to  develop  their  communicative  abilities  on 
an  electronic  network.  In  addition,  students  were  encouraged  to  participate 
actively  in  the  discussions  and  not  merely  be  recipients  of  information. 

The  Turkish  Case 

For  the  second-year  students  attending  Anadolu  University  in  Eskisehir, 
Turkey  who  participated  in  the  Global  Classroom,  communicating  via  the  EARN 
network  to  their  peers  on  another  continent  seemed  like  an  impossible 
achievement.  Many  of  the  students  were  from  Anatolia,  and  some  had  grown 
up  in  Eskisehir.  Their  knowledge  of  American  students  came  primarily  from 
their  exposure  to  American  television  programs,  and  they  were  anxious  to 
learn  more  about  the  conditions  and  way  of  life  of  their  fellow  students.  They 
were  eager  to  practice  their  English  skills  and  many  of  them  spent  hours 
reading  and  composing  messages. 

Besides  making  new  friends  and  practicing  English,  students  at  Anadolu 
were  eager  to  get  information  about  their  course  topic  which  was  not  available 
at  their  university  library.  Because  the  cost  of  an  imported  textbook  may 
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easily  be  half  a  month's  salary  of  the  average  worker,  students  and  even 
faculty  members  can  not  easily  purchase  foreign  text  materials.  Therefore, 
most  textbooks  used  at  Anadolu  are  written  in  Turkish  by  Turkish  professors 
and  many  of  those  are  translations  from  texts  published  outside  Turkey. 
Where  foreign  texts  are  available,  they  are  photocopied  and  distributed  to 
students  whenever  possible. 

Given  this  setting,  it  was  with  enthusiasm  that  students  approached  the 
task  of  learning  the  mechanics  of  getting  online  to  access  the  EARN  network. 
The  job  was  not  an  easy  one.  The  few  machines  at  the  university  which  were 
connected  to  EARN  and  available  to  students  were  constantly  in  use  and 
generally  in  need  of  repair.  Cookie  crumbs,  sesame  seeds  and  cigarette  ashes 
which  fell  into  the  keyboard  caused  keys  to  malfunction.  The  repair 
procedure  employed  by  the  computer  center  security  guard  was  to  turn  the 
keyboard  upside  down  and  beat  it  into  working.  In  addition,  the  university 
ran  out  of  coal  in  January  and  in  the  computer  laboratory,  an  unheated 
cement  building,  the  fingers  of  even  the  most  enthusiastic  students  would 
quickly  turn  cold.  Add  to  that  the  intermittent  electrical  failures,  keyboards 
with  sticking  letters,  the  temperature  in  the  room,  and  the  difficulty  of  access, 
it  was  remarkable  that  students  remained  enthusiastic  about  the  process. 

After  learning  how  to  use  the  EARN  network  and  being  given  the  topic 
for  discussion  of  that  week,  the  students  in  Turkey  would  discuss  the  issues 
among  themselves.  Because  the  semester  at  Anadolu  started  almost  two  months 
into  the  semester  in  the  United  States,  the  Turkish  students  were  at  a  double 
disadvantage.  Their  language  skills  were  not  at  the  level  of  the  university 
students  in  the  United  States,  and  they  came  into  the  computer  conference 
when  the  conference  was  one-third  over.  Nevertheless  they  were  fascinated 
with  what  the  technology  could  do  for  them  and  they  immediately  asked  for 
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library  articles  and  other  resources  which,  they  recognized,  the  other 
students  had  available  to  them.  Students  at  Anadolu  took  part  in  the  conference 
for  a  period  of  6  weeks.  They  did  not  participate  fully  due  to  time  of  entry  into 
the  discussion,  lack  of  fluency  in  English,  lack  of  adequate  resource  materials 
for  discussion,  and  inadequate  facilities. 

Results 

The  experience  at  Anadolu  University  indicates  that  there  are  a 
number  of  issues  which  need  to  be  addressed  when  the  electronic  classroom 
cuts  across  international  borders.  These  have  been  grouped  into  5  areas. 

First,  issues  of  pedagogy  must  be  discussed  in  advance  to  be  certain  that 
educational  assumptions  are  mutually  agreeable.  In  many  countries,  the 
prospect  of  electronic  communication  is  viewed  as  a  one-way  link  to  the 
outside  world  in  which  questions  and  requests  flow  in  one  direction  and 
materials  and  resources  flow  in  the  other.  In  addition,  many  countries  rely  on 
rote  learning  as  their  primary  pedagogical  principle.  Divergent  thinking, 
analysis,  problem  solving  and  group  discussion  have  not  been  desireable  in 
such  educational  programs.    If  the  expectations  of  one  group  participating  in 
the  global  classroom  are  not  similar  to  the  expectations  of  the  other,  the 
project  will  experience  some  difficulty. 

Mttlivjiligji  is  a  second  area  which  needs  to  be  examined  to  determine 
differences  between  students  in  various  cultures.  Both  intrinsic  and  external 
motivating  factors  are  very  diverse  in  the  International  community.  A  student 
in  one  setting  may  have  selected  a  course  of  study  where  another  student  may 
have  been  required  to  take  it. 

Access  to  facilities  which  provide  information  is  not  normally  available 
to  the  general  student  body  in  many  countries  as  it  is  in  North  America. 
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Information,  especially  when  provided  by  electronic  means,  is  frequently 
viewed  as  the  domain  of  the  faculty,  particularly  the  science  and  engineering 
faculty.  This  attitude,  that  the  use  of  technology  is  reserved  for  elite  faculty 
members  in  the  divisions  of  science  and  mathematics,  must  be  challenged  if 
students  are  to  have  access  to  information  for  making  well  informed  decisions. 

T>rhnical  difficulties  such  as  frequent  power  outages,  broken  down 
equipment,  lack  of  technical  support  and  a  large  percentage  of  downtime 
often  plague  those  students  who  most  need  to  be  encouraged  to  use  the 
technology.  As  a  result  of  these  technical  problems,  students  in  such 
situations  fall  farther  and  farther  behind  their  international  colleagues 
during  the  semester. 

Finally,  the  issue  of  language  i»  one  which  must  te  consldered  in 
international  classroom.  Although  students  in  many  international 
universities  read  and  write  English,  the  amount  of  written  material  which  is 
sent  electronically,  coupled  with  the  particular  Jargon  of  the  field  can  cost  the 
foreign  student  many  hours  of  deciphering.  Without  the  'face-to-face'  signals 
of  a  questioning  expression  or  a  vacant  look,  the  communicator  has  little  idea 
that  his  or  her  message  should  be  made  shorter  and  more  concise.  Most 
important  ideas  can  be  communicated  in  the  space  of  one  screen.  Part  of  the 
international  experience,  when  working  with  people  whose  English  is  a 
second  language,  is  to  simplify  language  and  concepts  so  that  the 
international  student  can  participate  in  the  exchange  of  information  rather 
than  merely  be  a  recipient  of  information. 

Conclusion 

In  spite  of  the  many  drawbacks  which  students  at  Anadolu  University 
encountered  in  this  experiment  participating  in  a  global  classroom,  students 
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who  joined  the  project  expressed  their  enthusiasm  for  computer-mediated 
communication  as  a  way  to  practice  their  English,  get  information,  draw 
informed  conclusions  about  topics  discussed,  develop  confidence  in  their 
ability  to  communicate  with  others,  make  friends  around  the  world,  and 
promote  world  peace.  In  short,  they  felt  empowered  with  new  skills  and 
abilities  to  access  information  and  participate  in  a  meaningful  exchange  of 
ideas  on  a  global  scale. 

As  INTERNET  links  join  more  global  classroom  projects  together, 
students  and  faculty  will  find  themselves  involved  in  meaningful  exchanges 
of  information  with  colleagues  around  the  world.  Library  resources  which 
are  so  limited  and  costly  will  soon  be  available  online  and  via  fax  to  remote 
locations  around  the  world.  As  we  begin  to  answer  the  question  of  how  to 
provide  equal  access  to  information,  we  must  look  ahead  at  what  kind  of 
information  will  be  shared.  Will  it  continue  to  be  information  by,  from  and 
for  the  dominant  cultures?  Or  will  computer-mediated  communication  truly 
empower  those  in  developing  countries  to  become  full  participants  in  the 
decision  making  process  of  the  global  community?  Therein  lies  the  challenge 
that  faces  implementation  of  the  new  telecommunication  technologies,  in 
order  to  help  people  achieve  their  full  intellectual  potential. 
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Abstract 

Qualitative  data  were  collected  from  28  homes  and  19  classrooms  at  four 
schools  participating  in  the  Buddy  System  Project.  Buddy  loans  each  student  and 
teacher  in  selected  elementary  classes  a  networked  home  computer,  and  provides 
the  schools  with  additional  classroom  computers.  In  this  paper  we  examine  three 
themes  from  the  cross  case  analysis:  the  impact  on  the  schools*  learning 
environments;  the  impact  and  use  of  the  telecommunications  functions;  and  the 
home  computers*  impact  on  the  home  environment 

Buddy  was  considered  a  success  in  all  of  the  schools.  The  computers  were 
enthusiastically  received  and  were  used  extensively.  There  was  high  use  of  tool- 
based  software  that  permitted  the  students  to  generate  and  process  information,  and 
a  minimum  use  of  the  computer  for  rewarding  behavior  or  for  rote  instruction.  The 
teachers  noted  that  the  project  had  a  renewal  effect  on  their  careers  and  a  positive 
impact  on  the  culture  of  the  classroom.  Teachers  also  reported  that  the  majority  of 
their  current  class  computer  activities  were  only  possible  because  there  were 
computers  at  school  and  at  home.  The  home  computers  allowed  the  students  to 
practice  and  explore  at  their  own  pace  and  permitted  teachers  to  assign  more 
involved  projects. 

The  Buddy  home  computers  have  contributed  to  general  dissemination  of 
computer  knowledge.  Students  entering  the  program  are  increasingly  familiar  with  the 
Buddy  computers  and  software  through  friends  and  siblings.  Parent  noted  that  their 
child  and  the  home  computer  helped  them  feel  more  comfortable  with  computer 
technology  and  terminology. 

Teachers  and  parents  reported  that  the  written  format  of  the  electronic 
communications  functions  has  had  a  positive  influence  on  the  students'  reading, 
writing,  and  typing  proficiency.  Students  often  preferred  spending  time  on 
BuddyNet  or  Prodigy  to  watching  television.  Teachers  reported  several  cases  of 
quiet  students  who  blossomed  on  e-mail,  initiating  communication  through 
extensive  e-mail  messages. 

A  significant  factor  in  successful  implementation  is  the  presence  of  several 
innovators;  teachers  or  administrators  ready  to  take  chances,  work  at  the  cutting 
edge,  and  lead  the  way  for  the  others  at  the  school.  In  addition,  the  success  of  the 
prograii i  would I  not  have  been  possible  without  a  large  amount  of  extra  effort  on  the 
part  of  the  teachers.  All  the  teachers  reported  spending  long  hours  learning  about 
software  and  trying  to  "keep  up  with  the  students."  In  all  cases  the  teachers 
reported  that  the  effort  was  well  worth  it. 
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Computers  Extending  the  Learning  Environment: 
Connecting  Home  and  School. 

What  gateways  to  learning  open  when  the  homes  and  classrooms  of  over 
2000  students  in  20  selected  elementary  schools  are  connected  by  networked 
computers  and  e-mail?  This  study  examines  qualitative  data  collected  fiom  28 
homes  and  19  classrooms  at  four  Buddy  System  Project  schools  to  provide 
answers  to  that  question. 

Buddy  equalizes  access  to  technology  by  loaning  each  student  and  teacher  in 
selected  Indiana  elementary  classes  a  home  computer,  modem  and  printer.  It  also 
provides  the  schools  with  approximately  one  computer  for  every  four  students  in 
the  classrooms.  In  addition,  it  seeks  to  expand  the  time  and  space  of  the  learning 
environment  by  linking  all  the  homes  and  schools  through  electronic  mail,  computer 
bulletin  boards  and  a  "chat  channel,"  and  providing  participants  access  to  a 
commercial  informational  network  such  as  Prodigy  or  US  VideoTel. 

In  this  paper  we  will  examine  three  themes  from  the  cross  case  analysis: 
Buddy's  impact  on  the  schools'  learning  environments;  the  impact  and  use  of 
telecommunications  functions;  and  Buddy's  impact  on  the  family  learning 
environment.  These  sections  arc  preceded  by  an  overview  of  the  Buddy  System 
Project,  an  abbreviated  literature  review,  and  a  methodology  section. 

The  Buddy  System  Project 

The  idea  for  the  project  arose  in  1987  when  members  from  the  private  sector 
working  with  the  Indiana  Corporation  for  Science  &  Technology,  met  with  H. 
Dean  Evans,  Indiana  Superintendent  of  Public  Instruction. 

Addressing  such  issues  as  education  in  the  Information  Age,  changing  work 
force  skills,  demands  on  telecommunications  infrastructure,  and  Indiana  s 
competitiveness  in  a  world  economy,  the  group  created  what  has  been  described 
as  the  most  innovative  educational  project  in  the  nation  today  (The  Buddy 
System  Project,  1991,  p.  1). 

The  long-range  vision  of  the  project  is  to  extend  the  learning  experience  beyond  the 
classroom  by  placing  a  networked  computer  in  the  home  of  every  fourth-  through 
twelfth-grade  student  in  the  state.  The  project  seeks  to  equalize  access  to 
technology  for  children  of  all  socioeconomic  backgrounds. 

Integrating  a  tool-use  approach  to  computers  into  the  curriculum  is  another 
of  Buddy's  goals.  The  project  emphasizes  teaching  students  to  use  the  computer  as 
a  tool  to  construct  knowledge.  In  order  to  encourage  this  approach,  the  project 
supplies  tool  software  such  as  word  processing  and  graphics  programs  and 
conducts  teacher  training  in  constructivist  computer  uses  and  learning  strategies,  in 
addition  to  providing  technical  support.  The  project's  major  objectives  fall  into 
three  categories.  These  include  improving  student  skills,  increasing  parental  and 
family  involvement  in  education,  and  providing  foundations  for  life-long  learning. 

The  program  piloted  in  the  fourth  grade  of  five  schools  in  1988.  By  the 
1991-92  school  year,  nearly  2,000  fourth-,  fifth-  and  sixth-grade  students  in  20 
schools  were  participating  in  the  Buddy  System  Project.  While  the  pilot  sites  were 
funded  predominantly  by  the  project's  grants,  additional  sites  were  funded  by  a 
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cost-sharing  plan.  TJe  latter  plan  was  based  on  a  commitment  of  40%  from  the 
school  district  and  60%  from  the  State  of  Indiana.  Funding  for  1991-92  included 
$1 .5  million  of  state  support. 

A  central  office  provided  technical  and  instructional  design  support  to  the 
schools.  A  help  Une  was  available  to  all  participants  for  any  technical  problems. 
Teachers  received  a  stipend  for  their  participation  and  were  provided  with  computer 
equipment  in  school  and  at  home.  Further  instructional  support  was  provided 
through  spring  and  summer  orientation  prior  to  fall  implementation,  source  books, 
monthly  workshops  and  newsletters,  as  well  as  periodic  school  visits  by  site 
coordinators.  Training  emphasized  that  teachers  were  not  to  teach  a  separate 
computer  class,  but  rather  to  integrate  computers  into  the  regular  curriculum. 

.  .     White  the  central  office  was  designed  to  provide  technical  and  instructional 
design  support  to  help  schools  attain  the  objectives  listed  above,  the  emphasis  of  the 
program  was  on  site-based  implementation.  While  the  five  pilot  sites  were  adopted 
by  either  IBM  or  Apple,  successive  schools  were  allowed  to  select  their  hardware 
&c    ^and  ^are  applications.  Schools  have  selected  IBM,  Macintosh,  Apple 
UOb  or  Commodore  Amiga  computers.  Software  applications  selected  included 
various  word  processing,  spreadsheet,  graphics,  and  communications  programs  as 
well  as  a  range  of  computer-based  instruction  programs  (CBI).  Each  school  also 
determined  the  configuration  of  the  computers.  Some  classes  pooled  all  of  the 
school-designated  computers  and  established  a  lab.  Others  opted  to  divide  them 
among  the  participating  classes.  m 

Te1emmmiini™rirni,i 

.  H,       classroom  and  home  in  the  Buddy  System  Project  has  access  to  a 
variety  of  telecommunications  functions.  These  functions  are  provided  by  two 
different  services  The  first  service  is  BuddyNet,  a  wide-area  network  that  was 
used  exclusively  by  Buddy  participants  and  their  families.  This  service  used  a 
switched-data  network  to  connect  all  participating  sites.  BuddyNet  provided  a 
variety  of  modes  of  communication  including:  one-to-one  and  one-to-many 
asynchronous  delayed-time  messages  via  electronic  mail  (e-mail)  and  bulletin 
boards;  and  one-to-one  or  many-to-many  synchronous  real-time  communication  via 
private  and  general  chat  channels. 

A  commercialized  information  service,  either  Prodigy  or  US  VideoTel  was 

f£^?f?i5,nCtT  availablc"  V™*  XTviccs  P""***  acces*s  to  educational  games 
and  simulations,  dictionaries  and  encyclopedia-like  data  bases,  as  well  as  current 
news,  entertainment,  sports,  and  weather  information.  Prodigy  also  allowed  the 
user  to  access  a  nation-wide  network  for  e-mail  and  bulletin  boards. 

Literature  Review 

10A7.  DN?,ta'ai!Sy8?s  of  computer  research  (Clark,  1983;  1985;  Kulik  &  Kulik, 

23?  ?°iblyC  CasUnc' &  K,ng' 1 988)  contend  *«     m»jo^  of  studies  which 
atmbute  learning  gains  to  computer-based  learning  environments  are  confounded  by 
coyarying  and  uncontrollable  factors.  There  are  two  primary  faults  found  in  most 
of  these  studies  The  first  is  that  the  studies  did  not  sufficiently  separate  ihe 
£22!?  instructional  method,  the  design  effort  invested,  and  the  content. 

Second,  s  udies  did  not: sufficiently  account  for  novelty  effects.  One  meta-analysis 

tSf1?^*  Ba?gcrt> &  Wil,iams>  1983>  f0«nd  *at  studiesEg 
four  weeks  or  less  had  almost  twice  the  treatment  effect  as  studies  lasting  between 
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four  and  eight  weeks,  indicating  that  novelty  is  a  major  confounding  factor  in  this 
research. 

There  is  a  need  to  supplement  the  outcome  measures  of  standardized  tests 
with  observations  of  die  learning  process.  In  this  way,  instead  of  determming 
whether  computers  influence  learning,  we  gain  a  better  understanding  of  how  the 
computers  influence  learning.  Qualitative  research  strategies  offer  a  preferable 
approach  to  understanding  how  computers  are  used.  In  addition,  the  researcher 
needs  to  be  cognizant  of  how  the  use  of  the  computer  interrelates  with  the  social 
dynamics  of  the  learning  environment.  Pedagogically  sound  classroom  computer 
use  depends  greatly  upon  the  teaching  style  employed.  Naturalistic,  field-based 
observational  techniques  provide  a  useful  approach  to  describing  different  teaching 
styles  and  their  relation  to  various  classroom  outcomes. 

Field-based  observational  techniques  were  used  in  a  study  conducted  to 
understand  the  impact  of  the  computer  on  classroom  organization,  teacher-student 
relations,  and  curriculum  (Mehan,  1989;  Riel,  1989).  In  this  study,  four 
elementary  school  teachers  were  each  given  one  Apple  H  computer  for  their 
classrooms,  and  their  classrooms  were  observed  over  the  course  of  a  year.  I  he 
researchers  found  that  the  teachers  fit  their  use  of  the  computers  into  existing 
temporal  and  spatial  classroom  organizational  practices.  Cooperative  student 
interaction  increased,  however,  when  teachers,  seeking  to  maximize  time  on  the 
computer,  had  students  work  in  pairs  instead  of  individually.  The  major  conclusion 
of  both  articles  from  this  study  was  that  classroom  organization  and  computer  use 
are  mutually  influential. 

Another  project,  Apple  Classrooms  of  Tomorrow  (ACOT)  is  striving  to 
document  how  instructional  innovations  emerge  in  high  computer  access 
environments.  ACOT  sites  have  high  computer  access,  readily  available 
technological  peripherals  such  as  video  discs,  and  an  emphasis  on  using  technology 
as  a  tool  to  construct  knowledge.  ACOT  has  emphasized  local  site  design  and 
control  of  the  high  access  environments.  The  project  did  not  implement  a 
standardized  program  of  computer  use,  but  encouraged  teachers  to  develop  new 
curricula  and  teaching  methods  in  order  to  fully  utilize  the  high  computer  access  in  a 
constructivist  manner. 

Teachers  were  provided  with  considerable  technology  supprat^but  were  not 
provided  with  support  for  effective  change  in  the  curricular  areas.  ACOT 
(•valuators  began  to  realize  that  teachers  were  likely  to  view  technology  resources  as 
they  viewed  other  instructional  materials;  as  additions  to  their  existing  repertoire  ot 
curricular  activities  instead  of  catalysts  of  constructive  student  work. 

We  suspected  that  the  omission  of  a  strong  curriculum  component  in  the 
majority  of  these  technology  projects  could  increase  such  risk.  To  explore  this 
possibility,  we  contrasted  technology  functions  in  two  sites  that  differed  in  the 
extent  of  their  integration  of  curriculum  design  with  technology  planning 
(Gearhart,  Herman,  &  Whittaker,  1991,  p.  4). 

Observations  indicated  that  although  the  tools  of  experts  (in  this  case  word 
processing  applications)  were  used  in  the  classroom,  they  were  not  necessarily 
employed  as  experts  would  have  used  them.  For  example,  while  all  the  teachers 
reported  that  students  were  more  motivated  to  write  using  the  word  processor  and 
were  proud  of  their  professional-looking  products,  not  all  of  the  teachers  were 
using  the  word  processor  to  support  the  expert's  process  approach  to  writing. 
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Thui,  while  students  were  learning  to  use  an  expert  tool,  they  were  not  necessarily 
learning  to  write  as  experts  write;  i.e.,  reworking  compositions  for  clarity  and  usine 
writing  as  a  way  of  learning.  6 

This  study  indicates  that  researchers  need  to  examine  more  than  just  the 
tools  that  are  used  in  the  learning  environment.  In  order  to  better  understand 
computer  "«*  in  education,  there  is  a  need  to  explore  the  ways  in  which  the 
variables  of  software,  instruction,  and  student  combine.  It  is  not  enough  to 
document  that  teachers  and  students  arc  using  tool  applications,  researchers  needs 
to  understand  how  these  tool  applications  arc  used  within  specific  contexts. 

Methodology 

_  _       y*csc  qualitative  case  studies  were  based  on  naturalistic  inquiry. 
Underlying  axioms  of  naturalistic  inquiry  include  the  beliefs  that: 

•  Realities  are  multiple,  constructed,  and  holistic. 

•  Knower  and  known  are  interactive,  inseparable. 

•  Only  time-  and  context-bound  working  hypotheses  are  possible. 

•  AH  entities  are  in  a  state  of  mutual  simultaneous  shaping,  so  that  it  is 
impossible  to  distinguish  causes  from  effects. 

•  Inquiry  is  value-bound.  (Lincoln  and  Guba,  1985,  p.37) 

Belief  in  multiple  realities  shifts  the  focus  of  research  from  prediction  and 
control  to  understanding  some  level  of  the  complexity  of  the  situation.  Thus,  these 
realities  must  be  studied  holistically  and  within  context.  This  approach  relies  on 
thick  description"  (Geertz,  1973).  The  rationales  for  this  type  of  approach  can  be 
n^mi£0ntcmE2?}?  «"c*«iological  writings  on  case  study  such  as  Donmoyer 

■SfiteoSi)       m       LincoIn  and  Guba  (1985)' MciTiam 

>u  J*? au?*  ?f  jimc  construi,,<!i « typology  was  established  to  set  boundaries  for 
the  study  that  included: 

•  Human  interface  with  the  technology:  What  ore  the  typical  uses  by 

students,  teachers  and  fumllles? 

•  Ptd^^y:  ^t'81  mc  relation  between  teaching  style  &  computer 

•  Organizational  structures:  Have  hierarchies  shifted  because  of  the 

project? 

•  Personal  efficacy:  What  are  low,  average  and  high  ability  students 

accomplishing  within  this  project? 

Starting  from  this  typology,  the  research  became  a  dialectic  between  foci 
and  emerging  themcsthat  the  participants  identified  as  important,  such  as  increased 
learning  motivation,  parent/teacher  communication,  and  a  tool-use  approach  to 
instruction.  Other  points  of  repeated  discussion  included  the  role  of  the 
telecommunications  functions,  the  students*  increased  self-esteem,  and  changes  in 
the  student/teacher  relationship.  This  report  was  prepared  to  provide  an 
understanding  of  three  areas: 

•  Buddy's  impact  on  the  school  learning  environment, 

•  Impact  and  use  of  the  telecommunications  functions, 

•  Buddy's  impact  on  the  family  learning  involvement. 
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fii»1i»r.rinn  nf  Sites 

Constraints  of  time  and  money  dictated  that  the  study  be  limited  to  four  of 
the  20  Buddy  schools.  Realizing  that  each  of  the  sites  had  a  uniaue  story  to  tell,  we 
understood  that  we  could  not  randomly  select  four  sites  that  would  provide 
generalizable  information.  We  purposefully  selected  diverse  sites  that  would 
provide  a  range  of  factors  involved  in  the  implementation  of  Buddy.  Areas  tor 
diversity  included;  hardware  platform,  length  of  involvement  in  the  project,  school 
size,  demographics,  and  socioeconomic  status.  The  four  sites  selected  were: 

.  Taft  Elementary  School1  -  A  rural  school  with  an  IBM  lab,  Taft  was  one 
of  the  original  five  pilot  schools.  The  school  provided  a  populauon  with  a  broad 
range  of  SES  and  no  racial  diversity  (100%  white).  The  program  included  two 
fourth-grade  and  two  fifth-grade  classes. 

•  Maplewood  Elementary  School  -  A  suburban  school  with  an  IBM  lab, 
Maplewood  was  one  of  the  original  five  pilot  schools.  The  school  is  a  Discovery 
Magnet  Program  school  with  an  emphasis  on  science.  The  school  provided  a 
population  with  a  range  of  SES  and  some  racial  diversity.  The  program  included 
four  fourth-grade  and  four  fifth-grade  classes.  Maplewood  was  slated  to  be  one  of 
the  first  Buddy  schools  to  make  the  transition  to  parent  buy-in  of  the  home 
computers. 

•  Plainview  Elementary  School  -  An  urban  school  with  Macintosh 
computers  in  each  classroom,  Plainview  was  in  its  second  year  of  Buddy 
involvement.  The  program  at  this  school  included  two  mildly  mentally  handicapped 
(MiMH)  classes,  one  fourth-grade  and  one  fifth-grade  class.  The  sixth  grade  was 
added  in  1992-93.  The  school  offered  a  population  with  a  low  to  medium  range  ot 
SES  and  some  racial  diversity.  This  site  was  also  identified  by  the  Buddy  office  as 
one  of  the  most  innovative  schools  in  its  use  of  computers. 

•  Linden  School  --  An  urban  school  with  Macintosh  computers  in  each 
classroom,  Linden  was  in  its  first  year  of  Buddy  involvement.  The  program 
included  two  fourth-grade  classes  and  one  LD  class.  It  will  expand  in  successive 
years  to  include  fifth  and  sixth  grade.  The  school's  populauon  offered  racial 
diversity  and  consisted  of  students  from  predominantly  low  SES  families. 

Family  Selection.  At  each  site  six  to  eight  families  were  selected  for  home 
visits  We  selected  a  range  of  student  characteristics  that  included  academic 
achievement  levels,  leveland  amount  of  computer  use,  socioeconomic  status,  race, 
and  gender. 

Calendar  of  Case  Work 

Two  researchers  were  assigned  to  each  of  the  sites.  The  teams  logged 
between  18  and  21  days  at  each  site.  Visits  to  homes  and  school,  some  in  tandem, 
some  individual,  were  spread  over  three  time  periods  during  the  1991-1992  school 


1  Anonymity  The  name  of  the  5ite  and  of  all  staff,  students  and  families  have  been  changedto 
protect  their  privacy.  In  some  cases,  non-consequential  identifying  features  were  also  changed 
to  further  protect  anonymity. 
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year:  two  to  three  days  in  late  November  and  early  December  of  1991;  four  days  in 
January  or  February  of  1992;  and  three  to  four  days  in  March  or  April  of  1992. 

Methods 

•  Open-ended  interviews  with  participants. 

•  Observations  of  schools,  homes  and  electronic  network. 

•  Review  of  program-,  school-  and  student-generated  documents. 

Interviews.  The  interviewing  methods  were  designed  to  allow  for  free- 
flowing  discussion  guided  by  the  established  foci  about  computers  use,  teaching 
styles,  classroom  interactions  and  instruction,  and  problems  encountered.  The 
relationship  between  home  and  school  was  also  discussed.  Probing  questions 
clarified  participants*  definitions  of  terms  and  provided  better  understanding  of  the 
meaning  of  responses.  Interviews  were  tape-recorded  and  then  transcribed  for 
analysis. 

The  student  interviews  were  conducted  on  a  one-to-one  basis  during  the 
school  day  and  were  approximately  twenty  minutes  in  length.  The  purpose  of  the 
interviews  was  to  determine  the  students'  perceptions  of  their  own  and  family  use 
of  the  computer  and  networic,  what  was  liked  best  and  least  about  the  project,  and 
how  school  assignments  had  changed  since  the  introduction  of  the  project 

The  teachers*  interviews  were  conducted  during  free  periods  and  lunch 
times.  These  interviews  were  more  extensive  than  the  student  interviews  and  often 
had  to  be  spread  over  several  days  because  of  their  length  and  the  lack  of  a 
continuous  block  of  time.  There  was  also  an  on-going  casual  dialogue  with  the 
teachers  throughout  our  visits.  Interview  questions  sought  to  explore  teachers' 
perceptions  of  changes  in  classroom  management  styles,  their  relationships  with 
students,  parents,  and  other  teachers  as  well  as  the  typical  and  optimum  uses  of 
computer  and  telecommunications  functions. 

Principals  and  the  site  coordinators  were  initially  interviewed  for 
approximately  an  hour.  There  were  follow-up  interviews  with  some  of  the  site 
coordinators.  Questions  focused  on  perceptions  of  organizational  and  pedagogical 
impact  of  the  computers  and  the  telecommunications  functions. 

The  interviews  with  families  took  place  in  the  homes  and  typically  took  an 
hour  and  a  half  on  the  first  visit  and  approximately  half  an  hour  on  the  second  visit. 
ihc  family  members  were  interviewed  as  a  group.  In  each  case,  the  Buddy  student 
had  previously  been  interviewed  individually  at  school.  In  these  interviews,  we 
attempted  to  determine  the  family's  use  patterns  and  perceptions  of  the  strengths 
and  weaknesses  of  the  project. 

Observations-  To  gain  a  better  understanding  of  teaching  styles  and 
approaches,  we  observed  teachers  instructing  both  with  and  without  the  computers. 
During  the  observations  of  computer  activities,  we  asked  the  teachers  pre-  and  post- 
observation  questions'  to  determine  their  perception  of  the  lessons'  objectives,  why 
they  had  selected  the  computer  for  the  particular  task,  and  how  successful  they  felt 
the  lesson  had  been. 

BuddyNet  was  observed  to  determine  the  content  and  continuity  of  the 
teachers  homework  postings.  Between  January  and  May  of  1992,  the 
announcements  on  the  electronic  bulletin  boards  of  the  schools  were  down-loaded 
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on  the  average  of  twice  a  month.  In  addition,  conversations  from  the  mam  chat 
channel  were  observed  for  ten-minute  intervals  at  random  times  of  day  on  the 
average  of  once  a  week.  Samples  of  these  exchanges  were  also  down-loaded  to 
provide  a  comparison  to  what  participants  reported  of  network  use. 

Qontcm  Review  »f  Documents.  Documents  were  collected  from  each  site. 
The  collection  included  the  site's  implementation  plan,  a  site  desenpnon  and 
student-generated  work.  Articles  written  by  newspapers  and  journals  about  the 
schools  and  about  the  Buddy  System  Project,  as  well  as  Buddy-generated 
publications  were  also  collected.  These  provided  background  ^formation  about  the 
schools  and  program  and  offered  further  means  of  understanding  what  was  being 
accomplished  with  the  computer. 

Credibility  and  Analysis.  To  increase  the  validity  of  conclusions,  the 
technique !of"iiangulation"(Denzir.  1978)  was  employed.  Triangularis  requmss 
data  be  substantiated  from  at  least  three  different  perspectives  before  it  is  submitted 
for  conclusion.  There  are  four  different  ways  to  triangulate  data:  use  of  muinple 
and  different  sources,  methods,  investigators,  and  theories.  This  study  utilized 
three  of  the  four  ways.  Two  investigators  were  assigned  to  each  site,  multiple  data 
sources  were  used,  and  data  were  collected  through  a  number  of  methods  including 
observations,  interviews  and  document  reviews. 

To  construct  the  cases,  the  collected  data  were  organized  as  a  function  of 
preparing  to  write.  Triangulation  and  participant  review  were  used  to  correct  errors 
of  fact  and  to  gain  general  confirmation  of  what  was  written.  Review  comments 
did  not  guarantee  revision,  but  consideration  was  given  to  how  the  suggestions 
could  help  provide  more  complete,  accurate  and  balanced  reports. 

The  School  Learning  Environment 

Computers  often  receive  lukewarm  receptions  from  teachers.  They  are 
unsure  of  how  to  use  the  computer,  in  both  a  practical  sense  as  well  as  a 
pedagogical  sense.  The  extensive  availability  of  computers  and  the  training  that 
Buddy  provided  were  identified  by  teachers  as  two  factors  that  produced  a 
noticeable  increase  in  the  level  of  computer  use  in  their  classrooms.  Though 
comfortable  with  the  technology,  few  teachers  in  our  study  articulated  a  special 
philosophy  of  computers  in  education.  The  majority  saw  computers  as  another  tool 
in  their  pre-existing  toolbox.  There  were  no  cases  of  changed  curriculum  due  to  the 
addition  of  computers.  Some  teachers  noted  that  they  were  able  to  go  into  more 
depth  on  the  subjects/but  felt  that  they  did  not  cover  any  additional  material  outside 
of  computer  literacy.  While  the  technology  did  not  revolutionize  their  approach,  it 
did  cause  the  teachers  to  think  about  their  teaching  approach. 

Computer  Configuration 

There  were  three  significantly  different  configurations  of  the  computers 
throughout  the  four  schools.  At  Plainview,  the  students  sat  at  tables  arranged  in  a 
U  and  the  computers  were  spaced  evenly  around  the  tables.  Thus,  the  computers 
were  always  available  for  use.  Furthermore,  by  their  placement  among  the 
students,  the  computers  were  positioned  to  be  used  in  cooperative  work  groups. 

At  Linden,  the  computers  were  lined  up  along  the  wall  of  the  classrooms. 
They  were  positioned  so  that  only  one  student  (two  in  a  tight  squeeze)  could  sit  at 
the  computer.  Thus,  while  the  computers  were  in  the  classroom  and  available,  the 
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class  had  to  be  disrupted  to  go  to  the  computers.  Hie  limited  space  meant  thev  had 
to  use  a  rotation  system.  This  individualized  approach  nOae^SSSSSSaSSL 
could  work  on  the  computer  and  limited  the  tyJEs  of  projects  in  Sy  enS 

t7Jc  °f.cr  two  icnools  placed  their  computers  in  a  separate  computer 
laboratory.  This  severely  limited  access  to  computers,  although  it  did  oermit  each 
student  to  work  at  his  or  her  own  computer.  *        ^  lcacn 
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.  f  .  **■         5  d!Sk*  of  a  sfParatc  computer  lab,  or  in-class  mini-labs  was 
am0U?  ofinte«ratcdLu8e-  T^hers  with  mini-lab  configurations  were 
KS  t0  u*  ^  computers  throughout  the  day.  Use  of  the  computers  at  the 
Knft?  W8V "P"5?*?  *•  *ne  ^signed  in  the  computer  laboratory  and  as  a 
£2  Au2  H*S/ariC"  fiequcnt;. i1*0ie  with  labs  viewed  computer  time  as  separate 
hZS i Sf-SJ  WCIt  ^  Ukclv  to  cmPhasize  individual  work  on  the  computer. 
22ul^  5Td  *cLuse    mc  computers  to  be  separate  from  the  remainder  of  the 
academic  activity  in  that  the  use  of  the  computers  was  not  discussed  untilthe 
£&E?  3SE  "T*  UP  to  «°  ^ lab-  While  a  configuration  might  reveal  apre- 
S2J5,.P5i08<Lphy,  lt  ^perpetuated  particular  uses.  Teachlrs  were  confined  to 
certain  approaches  given  the  physical  constraints  of  computer  access. 

Extending  irchnnlwnrfr.  The  teachers  emphasized  the  need  for  both  school 
computers  and  home  computers  to  facilitate  the  most  successful  use  of  the 
computer.  Without  the  home  computers  the  students  would  not  have  had  sufficient 
orneor  access  to  complete  projects  or  to  practice  drill  assignments.  As  one  teacher 

So  much  more  can  get  dons  at  home  where  the  kids  are  more  relaxed 
That  s  the  best  thing  about  having  them  at  home  for  me,  and  for  them 
judged^  Ca"  practlccs  what  mcv  nced  wit110"1  being  interrupted  or 

On  the  other  hand,  students  were  unable  to  progress  as  quickly  on  their  home 
computer  when  they  were  not  introduced  to  concepts  and  software  at  school 
ctlSUr?8 10  C  teachcre.  the  combination  of  access  was  especially  crucial  to  the 
students  progress  in  touch  typing  and  editing  functions.  These  were  viewed  as 
important  because  typing  facility  affected  the  students'  composition  writing. 

Level  nf  Prnficifitiny 

™m„„2!1CntUt^  "u  at  <U1/0ur  *chP°h  Possessed  a  basic  proficiency  with  the 
computer.  By  all  observations  and  interviews,  these  students  could  easily  turn  the 
computer  on,  load  software,  and  open  and  use  the  basic  operations  of  toe avaUable 

Sj^Sn!?  *  Llnd<S' *  f,m  ycar  Buddy  sch001'  ™re  *•  to^pSSS 
iSLZ? «     '  h0WCV4CJ; abt 10  opcn  Programs,  use  the  word  processor  and 
graphics  programs,  and  (for  the  most  part)  create  basic  HyperCard  stacks 

TypcnfUsp 

*  1,16  na^  of  ^  usc  of  the  computers  is  central  to  understandinn  their 
f2«       «oa  of  the  Buddy  System  Project  is  for  computers  to  be  viewed 

222^  1CTmg  10  **  COVeai  arcas'  IdcaUv'  toGl  »»  will  permit  the 
students  and  teachers  to  engage  in  activities  that  would  not  be  possible  without  the 
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computers  and  that  will  give  the  students  new  and  richer  insights  into  the  content 
areas.  The  schools  varied  in  the  degree  to  which  this  was  accomplished. 

We  identify  four  general  ways  in  which  computers  may  be  used  in  the 
school  learning  environment,  including  tool  use,  to  practice  content  or  stalls,  for 
computer  literacy,  and  as  a  reward  for  good  behavior.  We  use  these  classifications 
to  guide  our  discussion  of  the  uses  in  the  schools,  beginning  with  the  tool  use  that 
best  reflects  the  Buddy  philosophy. 

A«  a  tool  tn  develop  and  preset  information.  It  is  this  use  of  computers 
that  is  central  to  the  Buddy  philosophy.  The  computers  are  integrated  into  the 
instruction  and  serve  as  a  tool  for  working  in  the  instructional  domain. 

The  teachers  at  all  four  schools  promoted  the  tool  use  of  the  computers. 
The  teachers  emphasized  that  the  computer  not  be  viewed  as  being  for  games  but 
rather  for  work.  Most  often,  the  "work"  involved  a  substantial  project:  a  History 
Day  presentation;  a  submission  to  an  essay  competition;  a  proposal  for  a  space 
station.  The  teachers  noted  that  it  was  only  because  of  the  ready  availability  of  the 
computers  both  at  home  and  at  school  that  they  could  do  these  larger  tasks. 

This  philosophy  was  perhaps  most  fully  realized  at  Plainview.  Except  for 
the  typing  program,  the  use  of  the  computer  was  predominantly  used  for  tool  based 
software:  HyperCard,  graphics  programs,  and  Microsoft  Works  which  contains 
woid  processing,  data  base  and  spreadsheet  applications.  HyperCard  was  used 
extensively  to  create  presentations  in  science  and  social  studies.  However,  this  was 
not  just  a  substitute  for  paper  and  pencil  work.  Rather,  these  teachers  saw  the 
presentations  as  forcing  the  students  to  give  greater  consideration  to  the  audience 
and  to  the  use  of  original  rather  than  secondary  sources.  The  computer  permitted 
them  to  prepare  richer  presentations  by  incorporating  original  sources  such  as  video 
or  audio  clips.  For  example,  one  student  writing  about  the  first  African-American 
teacher  in  Plainview  sent  an  audio  tape  and  some  questions  to  that  teacher  who  now 
resides  in  California.  The  teacher  provided  the  answers  on  tape  and  those  answers 
were  digitized  and  included  in  the  HyperCard  stack.  The  science  teacher  at 
Plainview  talked  about  his  plans  to  use  public  databases  that  would  permit  the 
students  to  engage  in  real  scientific  activities.  In  these  examples  and  in  most  other 
instances  of  using  these  software  programs,  the  students  were  m  charge  of  deciding 
the  topic  and  designing  the  product.  The  teacher  served  as  an  advisor  to  help  the 
student  reach  his  or  her  goals. 

The  teachers  at  Linden  were  also  actively  searching  for  ways  to  integrate  the 
use  of  the  computer  into  the  curriculum  as  a  tool  for  learning.  They  spoke  of 
empowering  students.  Implementing  this  instructional  philosophy  proved  difficult. 
As  a  step  towaids  a  richer  learning  environment,  the  teachers  required  the  students 
to  develop  HyperCara  stacks  on  science  as  a  substitute  for  their  science  tests. 
While  we  see  this  as  a  positive  direction,  the  teachers  did  not  fully  realize  their  goal 
of  a  "richer"  learning  environment.  Rather  than  giving  the  students  freedom  in 
design,  and  serving  as  advisors,  the  teachers  maintained  considerable  control  over 
the  design  and  content  of  the  stack.  The  number  of  cards  and  the  content  of  each 
was  prc-specified.  In  another  project,  the  HyperCard  stack  had  to  be  produced  on 
paper  and  submitted  to  the  teacher  for  approval  before  being  typed  into  the 
computer.  As  a  consequence,  the  assignments  degenerated  into  worksheet  type 
exercises.  It  was  more  fun  for  the  students  and  it  engaged  them  longer,  but  it  was  a 
worksheet  nonetheless.  As  the  site  coordinator  at  this  school  noted,  there  is  a  need 
to  help  "teachers  change  attitudes  about  how  information  is  learned  and  how  to 
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impart  information."  They  need  to  move  away  from  the  old  paradigm  of 
information  transmission  by  an  expert  teacher. 

At  Taft  and  Maplewood  the  teachers  talked  about  the  computer  permitting 
them  to  work  on  large  projects;  projects  that  would  not  be  possible  without  a 
computer.  As  one  teacher  noted: 

Most  of  the  time  what  we  do  on  the  computer  has  many  steps  to  it. 
There's  a  purpose  for  every  one  of  them,  there's  a  sequence  to  it,  and 
there's  a  goal  at  the  end  You  wouldn't  be  able  to  do  that  if  you  didn't 
have  a  computer.  It  would  be  too  time  consuming,  too  frustrating. 

As  an  example  of  this  type  of  activity,  she  noted  that  they  were  preparing  a  proposal 
for  the  creation  of  a  space  station  using  the  word  processor  and  graphic  program. 
The  proposal,  when  completed,  would  be  about  forty  typed  pages. 

c  Provide  nracticc  in  a  content  or  skill  domain  Here  we  are  referencing  the 
of  educational  games  as  well  as  drill  and  practice  programs.  This  includes 
drilling  the  student  in  typing,  math,  spelling,  and  grammar.  It  also  includes 
educational  games  such  as  "Where  in  the  World  is  Carmen  San  Diego?"  where 
students  practice  problem-solving  skills  and  information-seeking  skills. 

At  Taft  and  Maplewood,  the  students  regularly  worked  with  spelling, 
arithmetic,  grammar,  and  typing  programs.  Based  on  our  observations,  it  would 
seem  ihat  40%  of  the  computer  time  was  spent  on  these  activities.  At  Linden,  drill 
and practice  software  was  used  less  frequently,  but  the  tool  applications  were  often 
utilized  in  a  drill  and  practice  manner.  For  example,  the  word  processor  was  used 
to  type  in  the  weekly  list  of  vocabulary  words  and  then  erase  them  as  a  means  of 
practicing  typing  and  vocabulary  words  at  the  same  time.  In  contrast,  Plainview 
used  little  drill  and  practice  programs  in  the  regular  classrooms  except  for  the  use  of 
a  touch  typing  program  and  a  math  "game." 

u  u  ™  ^  311(1  Practice  m  frequently  used  with  the  special  education  students  at 
both  Plainview  and  Linden.  The  repetitive  nature  and  the  immediate  feedback  offered 
by  these  programs  is  viewed  as  especially  appropriate  for  the  limited  attention  span  of 
these  students.  r 

Computer  literacy.  A  focus  on  computer  literacy  is  geared  to  having  the 
students  learn  about  computers.  This  entails  teaching  what  a  computer  can  be  made 
to  do  separate  from  other  instruction  instead  of  using  the  computer  as  a  tool  for 
working  within  a  content  domain.  In  general,  we  found  a  tendency  to  focus  on 
computer  literacy  as  a  goal  when  the  teachers  were  struggling  to  integrate  the  use  of 
the  computers  into  instruction.  For  example,  when  having  students  create 
databases  using  a  spreadsheet  or  database  program,  there  was  typically  little 
concern  with  the  content,  the  goal  was  to  give  the  students  experience  with  the 
mechanics  of  the  software  We  think  that  instruction  would  be  more  meaningful  to 
the  students  and  more  efficient  instructively  if  the  use  of  the  database  involved 
helping  the  students  ask  more  meaningful  questions  within  the  content  areas  thev 
were  studying.  3 

All  four  schools  needed  to  provide  a  computer  literacy  orientation  to  their 
incoming  fourth  graders.  For  the  most  part,  the  entire  fourth-grade  year  is  a  time 
when  the  students  learn  to  use  the  tools,  and  by  fifth  grade  the  students  are  able  to 
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use  them  more  as  tools  for  work.  Several  of  the  goffer  summer  programs 
before  the  fourth-grade  year  to  teach  basic  computer  literacy. 

In  the  spools  that  have  been  implementing  Buddy  for  several  year •now,  it 
is  becoming  r*S  *e  basic  computed  literacy  is  disseminating  to  jwdentseven 
iSSTJ TmS  Xcfourth  grade.  As  many  as  half  of  the  incoming  fourth-grade 
SSSsct^Sate  because^  their  exposure  to  neighbors  and 

siblings'  Buddy  computers. 

To  minforceBOodJjchayioI.  When  classroom  access  to  computers  is 

SSf  One  S»S7£ KTlHm  class  where  if  students  did  computer 
htn^work^ey^reScuSd  from  ten  minutes  of  math  class  so  ttat  they  «>uld  do 

learning  other  subjects.  This  approach  is  contradictory  to  die  mtegrative  tool 
atthude  toward  computers  that  Buddy  is  trying  to  encourage. 

Srif-Esteem 

An  overwhelming  response  from  the  teachers  at  all  four  sites  was  that  the 

wmmmmm 

It  SfSta TM.  increased  self-esteem  translated  into  greater 
enthusiasm  for  school. 

Equalising  students 

*rue  comnuters  played  a  particularly  important  role  for  those  students  with 

2K  mnXonalitv  and  new  applications  of  the  programs.  Everyone  could 
readily  contribute. 

On  a  broader  basis,  and  with  much  greater  implications,  the  computers  seem 
to  provide  partteular  support  to  students  with  learning  disabilities.  S^ents^mth 

v tew  memsehel  One  teacher  noted  that  their  abilities  on  die  computer  had 
"reSrammed"  her  expectations  of  what  they  were  capable  of  doing  Another 
teSS  *at the  development  of  computer  skills  by  students  in  the  special 
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MRU? SSS.  *~ 10  *  mainstrcamed  and  e™  excel  in  the  computer 

™.  ^.2*  ^P^1  of  ^  computer  technology  on  the  special  needs  students  was 
one  of  the  greatest  success  stories  that  we  observed.  There  wercthrceTuch clatL 

JSKLT0  d "TV  at  ^lainvicw  with  studcnts  Ossified  a^mMy 

«l2S3ff  ife  daSS^m  at  HndBn  Wherc  studcnts  wcrc  ^sified 
S JS^XiS^S?  •       a11  ^  of  ^  c,asses     teachers,  students,  and 
parents  ex  oiled  the  impact  on  the  students.  They  were  learning  more  thevwere 
remembering  more,  and  they  were  getting  more  practice  aThTesShke 

Sru?e7roagZ(s)SU,g  tel^mmunicati<™>  and  problem  solving  (learning  how  to 

Cooperative  I^arninp 

The  Buddy  System  Project  promotes  cooperative  learnine  In  some  school* 
where  this  was  already  the  instructional  model,  the  computeSmed  to  enhSc? 

and  Sha^nf •  ?vcn  whcre  this  modcl  was  nm  operXeXre  waTa 
noticeable ^increase  in  sharing  among  students.  Teachers  atSfL  schools  told 

EnerpTinptheTeachfti^ 

.  .    ,  W™16.  the  teachers  emphasized  the  enormous  amount  of  extra  work  reauireri 
LTJftS?ttng  *?  Budd/  E110^'         ■»  «*°  ™ted  that  the  pra^^ 
Snf  ^SiiT?  the"lt0  rcthink       cuniculum  and  thcffiwSdi  to 
t&to^^JSlv**"  T  ntW  °PP°^ties  in  learningPand  saw  me 
students  in  a  new  light  and  this  seemed  to  have  an  energizing  effect  This  is 
expressed  most  succinctly  by  a  teacher  of  mildly  naJRSSiJdnStm 
She  reported  that  during  the  first  year  of  the  program  ^^S^SS.SlSS'm 

Perhaps  we  in  education  have  unknowingly  limited  [our]  students  bv 
prejudging  what  they  can  and  cannot  do6  With  the  Buddy  sjsteml 
jJj  n°  ^ow  en°ugh  to  limit  them  with  my  expectations  since  I  truly 

Hi  TJT"  Wh8t  t0  ^F*1  ■  • In  actuality  thiVbecame  a  very  happy 
and  fun  classroom,  unlike  anything  any  of  us  had  experienced  I 
don  t  know  who  gained  more  in  these  experiences,  students  S  teacher. 
Culture  of  the  Classroom 

Clearly  a  change  of  the  magnitude  of  the  Buddy  Svstem  Project  mav  tv» 
expected  to  impact  the  culture  of  the  classroom  by  afi&oStte  3w 
meracts  and  the  way  everyone  thinks  of  learning^d Caching  We  saw  ome  of 

ex«r^ 

ffiv£w?S 7  u™d  begaiIi0  rcly  on  mis  ex^se  in  solving  problems 
Buddy  helped  shift  teachers  away  from  the  model  of  the  authoritarian  Whilewe 

fflavWflr!?tflh0ntarian  tCach?nSit  is  clear  that  the  stance  wS^^le  we 
Krs^ 
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All  of  the  teachers  and  most  of  the  students  and  parents  we  talked  to,  were 
aware  of  the  change  in  relationship.  Rather  than  resisting  the  loss  of  power  and 
SSSX S«  seemed  to  bask  in  the  glory  of  the  developing  expemse j  of 
STtudents.  It  was  exciting  of  them  to  see  the  students  being  able  tc .teach them. 
It  allowed  them  to  see  the  students  in  a  new  light,  recognizing capabilities  far 
teyondlvhat  Sey  had  previously  believed  possible.  In  a  similar  vein  the  students 
S£ri3  the  role  oAeache/  with  all  of  its  problems  and  work  and  witnessed 
first  hand  that  learning  is  a  lifetime  experience. 

Willingness  to  work.  All  of  the  teachers  and  students  reported  that  working 
on  the  SKSun.  It  was  generally  agreed  that  the  students  were  more 
willing  to  do  school  work  and  homework  on  the  computer  and  were  more 
en  husiastic  about  doing  it.  This  may  well  have  been  a  temporary  effect  due  to  the 
compuS^        however,  we  did  not  see  the  effect  significantly  subsiding 
across  grade  levels  or  in  schools  with  four  years  of  involvement. 

Tmnrrwwl  writing  pnH  composition.  All  teachers  reported  that  using  the s  word 
nrocessonrE  seen  in  the  amount  written  and  m  the 

dSpS  nthewriSng.  Several  teachers  noted  that  students  are  no  longer 
asK  C  long  Sic "essay  had  to  be,  but  rather  are  asking  how  long  they  could  make 
h  tateSttly  it  did  not  mean  that  spelling  was  better  even  though  a  spell  checker 
wJ ^ffiyit  rtiSS^  universally,  told  us  that  they  liked  writing  on  to 
computer because  it  did  not  hurt  their  fingers  like  writing  did.  It  was  easier  for  them 
?o^te  more,  so  they  did.  The  professional  appearance  of  the  printed  papers  has 
raised lelf-esteem,  especially  for  low  ability  students  The  increased  legibility  also 
helped  students  proof  read  their  own  papers  more  effectively. 

Problems 

We  identified  two  significant  problems  in  the  use  of  the  >mputer  in  the 
classroom.  The  first  was  initial  student  training.  The  fourth-grade  teacters  have 
spent  too  much  time  teaching  computer  mechanics  at  the  beginning  of  the '  year. 
While  this  has  been  a  problem,  aU  of  the  schools  have  already  generated  their  own 
solutions.  Maplewood  has  designated  the  whole  fourth-grade  year  as  the  ame  to 
develop  computer  literacy.  The  other  schools  have  initiated  a  Buddy  summer 
school  in  which  the  new  fourth-grade  students  received  training  m  the  P™^ 
applications.  The  second  problem  has  been  maintenance  of  hardware  and  of  local 
area  networks  where  most  of  the  software  was  stored. 

Telecommunications 

When  the  students  were  asked  what  the  Buddy  Project  meant  to  them,  they 
overwhelmingly  responded  "BuddyNet."  This  electronic  network  was , used  to  post 
homework  assignments  and  Brain  Teasers,  to  conduct  a  summer  Buddy  Program 
(at  Taft),  and  to  communicate  with  people  from  all  over  the  state. 

Empowerment 

One  of  the  most  exciting  telecommunications  observations  is  its  potential  to 
empower  diverse  groups  (low  ability,  rural,  minorities).  The  anonymity  of  the 
communication  minimizes  such  diverse  and  often  segregating  conditions  as  gender, 
race,  physical  appearances  and  socioeconomic  status.  In  one  rural  area  tnat  nas  a 
reputation  for  racism,  the  program  coordinator  reported  that  this  electronic 
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^dSdTversi^^     Students      Positive  3X1(1  otherwise  unlikely  exposure 

At  another  school,  the  students  come  from  a  background  that  gives  them 
umited  expectations  of  their  future.  This  is  reinforced  by  both  economic  and 
cultural  factors  BuddyNet  provides  an  opportunity  for  interaction  beyond  the 
boundaries  of  their  immediate  environment.  It  allows  them  to  communicate  with 
students  from  other  schools  and  provides  the  potential  for  exposure  to  new 
Sf.reJKV?S.  and,518her  expectations.  Already,  teachers  have  noticed  a  difference  in 
the  students  spelling  and  grammar  as  they  strive  to  communicate  in  this  new 
environment. 

Social  OiiflPt 

finH  »h.2uddyN?tuals^*^fles  m  environment  where  socially  awkwaid  students  can 
Si?7  r  mchC-  7P*C  man-y  carlv  comPuter  studies  raised  concerns  over  the 
deserialization  caused  by  excessive  computer  use,  BuddyNet  had  quite  the  oDDosite 

5f      ^qmCVStuU^nts  often  prcferred  this  vchicle  to  communicate  with  teachers 
and  with  others  of  their  own  age.  The  network  hides  physical  imperfections  and 
personality  flaws  which  frequently  serve  to  isolate  students  in  the  classroom. 

u Sev?al  tea^e=rs  ^P0"^1  cases  where  e-mail  opened  channels  with  students 
with  whom  it  was  difficult  to  communicate.  They  noted  that  painfully  quiet 
students  who  they  were  unable  to  convince  to  speak  in  class  often  initiated 
communication  through  extensive  e-mail  messages.  New  students  or  those  who 
having  problems  were  subtly  and  easily  "touched"  in  short  e-mail  messages. 

The  Mall  SvnHmmr 

„, .»    ™n  Chat  ^  C"mai.1  ?aPacities  also  provide  a  place  for  the  students  to  "hang 
SUSST*  ii"    S  Way  U 18  singly  similar  to  the  teenage  "mall  syndrome." 
™,m    ,WOud  £F",?TBI,fS 10  mcct  each  other> or  would  telephone  to  see  if  a  friend 
could  get  on  BuddyNet.  The  extended  network  connected  students  to  people  in 
different  areas  of  the  state  and,  for  those  with  Prodigy  capacity,  the  country. 

Time  On-line 

Students  often  preferred  spending  time  on  BuddyNet  chat  or  Prodigy  to 

°Ur      showl!ittle  P^tal  concern  with  the  amouSof  time 
SSLw  ^hc  C0,m4)utcrn0_r  m  ^  0{  prodigy  which  shows  more  advertising 
than  Whittle  s  Channel  One.  The  interactive  nature  of  the  computer  made  it  more 

Jffi!«g^?T!parcntS  and  ***  communication  aspect  made  it  more  appealing  to  the 
students.  There  were  some  complaints  of  extensive  tie  ups  of  phone  lines,  but 

ISS^^}0  uaVCu^rkf?  out  Ume  schedules,  or  were  resigned  to  the  belief 
that  the  benefits  to  the  child  of  being  on-line  outweighed  the  inconvenience. 

Rules  nnH  Rt»ffli1nfjnn.a 

„«  ne  nTaJi'  mT°rc  ^°,than  any  other  ^hool  staff,  "chaperoned"  the  students'  social 

Z  lSffi?ei         was  a  W**  Curfcw  **  foiSl  S™1™  wcrc  restricted  ra 
™LE  a. J  it  chann*ls;  S60?.™1  chat  was  something  they  graduated  to  in  the  fifth 
grade.  Adults  were  often  on-line  to  remind  students  to  place  a  period  at  the  end  of  a 

SlnnCi^ 10  X™  as  a  P^sfnce  *at  encouraged  a  higher  standard  of  interaction. 
Linden,  in  comparison,  had  few  rules  and  no  curfew. 
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Facilitating  Home-School  Communication 

The  e-mail  function  had  a  variety  of  levels  of  uses  for  facilitating 
communication  between  home  and  school.  The  most  common  "use"  was  the 

Eerception  that  the  lines  between  home  and  school  were  more  accessible.  Because 
omework  assignments  are  often  listed  on  the  school  bulletin  boards,  it  is  difficult 
for  students  to  tell  their  parents  that  they  have  no  homework,  and  teachers,  parents 
and  students  credit  this  factor  for  the  increase  in  assignment  completion. 


Personal  Invitation 

Teachers  found  that  when  the  students  read  their  assignments  over  the 
computer,  they  were  more  likely  to  be  motivated  to  complete  them.  One  teacher 
theorized  that  perhaps  his  requests  seemed  more  personalized  when  they  were  seen 
on  the  students'  home  monitor. 

I  think  standing  in  front  of  the  class,  they  can  see  that  you're  talking  to 
everyone.  I'm  not  sure  when  they  read  it  on  their  computer  that  they  realize 
that  you  wrote  that  same  message  to  everyone.  I  think  if  you  asked  them 
about  it  they'd  understand  that,  but  somehow  because  it  is  one  letter  to  them 
in  their  house  it  is  personalized;  and  then,  they  had  to  write  back  to  me 
personally. 

RmM  .ifr.  Applications 

BuddyNet  provided  real-life  opportunities  to  communicate  in  the  written 
form.  Teachers  emphasized  that  this  writing  practice  was  more  likely  to  occur  in 
the  e-mail  mode  than  in  the  chat  channel.  In  e-mail  and  on  the  bulletin  boards  the 
students  were  more  likely  to  write  full  sentences  and  paragraphs.  For  example,  one 
principal  received  an  unprompted  e-mail  note  from  a  student  protesting  the  school 
ban  on  remote  control  cars.  What  pleased  the  principal  most  about  the  letter  was  the 
student's  initiative  and  the  real-life  purpose  behind  the  communication. 
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Tutoring  On-line 

The  electronic  functions  opened  some  unexpected  gateways  to  learning. 
One  student  with  a  severe  reading  learning  disability  was  "adopted"  by  a  woman  in 
another  community.  The  two  met  on-line  every  night  and  through  "chatting" 
improved  the  girl's  reading  and  writing  skills.  Adults  noted  that  the  written  format 
of  the  highly  popular  chat  and  e-mail  has  a  positive  influence  on  the  students' 
reading  and  writing  proficiency. 

Summer  School 

Taft,  the  rural  school,  conducts  its  summer  Buddy  program  on-line.  Prior 
to  the  introduction  of  the  Buddy  System  Project,  none  of  the  teachers  at  Taft  taught 
summer  school  classes.  The  school  building  was  closed  for  the  summer.  Any  Taft 
student  who  wanted  to  participate  in  summer  programs  (for  either  enrichment  or 
remedial  reasons)  had  to  be  driven  into  the  nearest  town,  which  is  almost  twenty 
minutes  away.  In  contrast,  the  on-line  learning  environment  created  in  BuddyNet  is 
easily  accessed  and  can  be  attended  to  at  a  time  that  is  convenient  to  each  individual 
student  and  teacher.  The  on-line  projects  allow  and  encourage  all  levels  and  abilities 
of  students  to  stay  involved  with  learning  over  the  summer  months.  They  reinforce 
what  had  been  learned  the  previous  year,  explore  new  topics  and  provide  a  way  for 
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teachers  to  become  more  familiar  with  the  students  and  their  needs  before  the  class 
physically  convened. 

Problems 

Forgotten  or  mis-entered  passwords  were  a  frequent  source  of  confusion 
tor  new  users.  Mis-loaded  software  also  prevented  many  new  users  from  logging- 
on  for  several  weeks,  by  which  time  they  had  forgotten  the  procedure.  Using  the 
telecommunications  feature  rendered  the  family  line  busy.  If  the  family  had  call 
waiting,  an  in-coming  call  would  bump  the  user  off  of  the  network.  Both  of  these 
situations  were  considered  nuisances  by  the  participants. 

Family  Involvement 

pis  section  concentrates  on  the  information  culled  from  home  visits 
Research  of  home  use  of  personal  computers  is,  to  date,  lacking.  One  goal  of  the 
Buddy  System  Project  is  to  support  the  entire  family  in  developing  computer  skills 
and  in  integrating  the  computer  into  their  daily  lives.  Home  visits  and  family 
interviews  provided  insight  not  only  into  family  computer  use,  but  also  into  the 
home  computer's  role  in  the  child's  education. 

Use  Of  thft  rnmpnt^r 

UscbyParcms,  The  two  most  popular  activities  among  the  parents  were 
the  use i  of BuddyNet  and  Prodigy.  The  chat  channel  was  by  far  the  most  popular 
use  of  BuddyNet.  Parents  and  children  alike  spent  long  hours  simply  talking  with 
other  people  on  the  system.  Most  often  this  was  general  social  chatter.  However, 
the  effect  was  not  always  neutral.  For  example,  through  the  Buddy  chat  channel 
one  parent  learned  of  another  parent's  problems  with  a  water  heater  on 
Thanksgiving  Day  and  was  able  to  give  advice  over  BuddyNet  on  how  to  make  the 
repau-.  At  the  other  end  of  the  spectrum,  there  have  been  numerous  instances  of 
foul  language  and  flirting  by  the  parents  on  BuddyNet. 

There  were  some  instances  of  the  use  of  the  Buddy  computer  to  help  with 
work-related  tasks,  but  this  was  infrequent.  Most  often  the  computer  was  used  as  a 
word  processor  to  generate  letters,  newsletters,  or  announcements.  One  mother 
used  the  system  to  keep  records  from  her  job  as  a  nurse.  However,  the  removal  of 
the  computer  for  the  summer  inhibited  her  from  continuing  that  activity  when  the 
computer  was  returned  that  following  school  year.  Several  parents  noted  that  they 
used  the  BuddyNet  to  contact  and  communicate  with  the  parents  and  members  of 
the*  scout  troops.  Another  parent  told  us  of  a  school  committee  meeting  that  was 
held  on-line  because  it  was  more  convenient  and  time  efficient  than  driving  to  a 
common  meeting  place. 

Often  we  heard  about  parents'  plans  to  use  the  computer  -  plans  that  never 
seemed  to  be  realized  For  example  one  parent  who  is  in  sales  indicated  he  intended 
to  use  the  computer  to  maintain  a  client  database.  However,  when  we  returned 
three  months  later,  he  told  us  that  he  had  not  done  anything  yet  because  he  had  long 
hours,  as  well  as  a  lot  of  travel  with  the  job.  Another  parent  indicated  that  she 
would  like  to  use  the  computer  for  family  budgeting,  while  a  thiid  indicated  that  he 
saw  it  as  a  mechanism  for  managing  his  stamp  collection.  Finally,  one  parent  saw 
Uie  computer  at  home  as  making  it  easier  to  take  a  computer  class  that  would  help 
her  develop  the  computing  skills  necessary  for  secretarial  work.  However,  she  was 
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still  too  unfamiliar  with  the  computer  and  it's  uses  to  be  able  to  express  just  what 
skills  she  needed. 

TTsfthv  siblings.  The  use  by  the  brothers  and  sisters  of  the  Buddy  child  is 
dominated  by  games  and  BuddyNet.  The  children  do  not  seem  to  distinguish 
between  educational  and  non-educational  games.  That  classification  seems  to  be 
one  that  we  as  adults  impose.  Graphic  programs  are  also  popular  with  younger 
siblings,  but  generally  not  with  older  siblings.  It  is  interesting  to  note  that  the 
siblings  in  the  families  we  interviewed  did  not  use  the  computer  for  their  ( 
homework.  This  was  true  even  with  siblings  who  were  "Buddy  alumni.  This 
reinforces  the  notion  that  the  effective  use  of  the  computers  requires  the  creation  of 
a  culture  for  its  use. 

Buddy  home  computers  have  contributed  to  general  dissemination  of 
computer  knowledge.  Students  entering  the  fourth  grade  are  increasingly  familiar 
with  the  buddy  computers  and  software;  even  those  who  have  not  had  a  computer 
in  their  home  yet.  At  one  school  in  its  fifth  year  of  implementation,  approximately 
half  of  the  members  of  the  incoming  fourth-grade  class  were  familiar  because  of 
their  exposure  to  neighbors'  and  siblings'  Buddy  computers.  Parents  noted  that  the 
home  computers  had  made  them  more  comfortable  with  and  aware  of  hardware, 
software,  and  the  potential  held  by  the  development  of  computer  related  skills,  such 
as  word  processing. 

Homework 

Parent  involvement  in  homework.  In  many  cases  there  was  no  change  in  the 
parents'  level  of  involvement  with  homework.  However,  across  the  schools  we  did 
see  three  types  of  effects.  First,  the  newness  of  the  technology  and  the  child  s 
expertise  attracted  the  parents  to  the  activity.  They  would  share  the  homework  time 
with  the  student,  learning  the  technology  as  the  student  did.  Second,  the  activities  of 
the  school  required  the  parents  to  be  more  involved.  This  was  the  case  at  Plainview. 
Parent  involvement  remained  high  as  long  as  there  was  a  requirement.  Finally, 
some  parents  were  frustrated  in  their  attempts  to  work  with  the  child  because  they 
did  not  know  how  to  use  the  software  and  hence  could  not  provide  help. 

TVaHng  with  divorce.  The  effective  use  of  the  computer  for  homework  is  a 
complex  issue  because  of  the  structure  of  family  units  in  our  modem  society. 
Specifically,  a  large  proportion  of  the  students  in  three  of  the  schools  come  from 
families  where  the  parents  are  separated  or  divorced  but  with  both  parents  still 
living  in  the  same  general  area.  As  a  consequence,  the  children  actually  reside  in 
two  homes;  the  mother's  and  the  father's.  Their  access  to  the  home  computer 
depends  on  whose  home  they  are  in  for  the  evening.  The  teachers  have  used  three 
strategies  to  manage  this  problem.  First,  use  of  the  computer  for  homework  is 
made  optional.  Students  may  do  it  on  the  computer  or  not,  as  they  see  fit.  Second, 
the  computer  homework  is  a  means  of  achieving  extra  credit.  If  the  homework  is 
done,  then  bonus  points  or  some  other  reward  is  given.  Third,  the  student  is  given 
time  to  work  on  the  computer  homework  assignment  at  school. 

Parent-School  Linkages 

We  found  that  while  parents  agreed  that  the  program  improved 
parent/school  relations,  they  did  not  perceive  as  great  an  improvement  as  the  school 
staff  had.  Both  recognized  that  the  logistics  of  getting  the  equipment  home  required 
an  increase  in  the  number  of  contacts  between  teachers  and  parents.  Because  these 
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contacts  were  for  an  action  considered  to  be  neutral  or  positive  (instead  of  a 
negative  confrontation  over  the  child's  grades  or  behavior)  teachers  and  parents 
noted  that  they  felt  communication  had  improved.  Perhaps  the  electronic 
communication's  greatest  "usefulness"  was  not  an  increase  in  actual 
communication,  but  rather  an  increase  in  the  perception  that  the  lines  of 
communication  between  home  and  school  were  more  accessible. 

Buddy  also  facilitated  the  opportunity  for  school  staff  to  visit  homes. 
Setting  up  the  home  Buddy  computer  sometimes  created  a  need  for  a  visit  from  a 
school  staff  member.  Taking  the  initiative  to  meet  the  family  on  their  "own  turf* 
under  positive  circumstances  was  well  received  by  the  families.  Through  seeing  the 
home  situations  of  students,  the  teachers  were  able  to  gain  a  better  understanding 
of  behavior  observed  in  school 

The  BuddyNet  e-mail  and  bulletin  board  functions  are  also  used  to  facilitate 
communication  between  home  and  school,  as  well  as  to  get  parents  more  involved 
in  their  child's  education.  Teachers  who  actively  established  improved 
communication  with  parents  as  a  high  priority  used  successful  tactics  such  as 
requiring  parental  e-mail  each  night  upon  student  completion  of  homework.  Other 
teachers  post  assignments  that  require  the  involvement  of  family  members.  The 
posting  of  "brain  teasers"  at  one  school  has  become  a  well  received  family  activity. 
The  program's  emphasis  on  family  involvement  has  received  mixed  reviews  from 
parents.  Some  felt  that  these  assignment  checks  were  an  imposition,  others,  often 
those  who  were  already  involved,  were  pleased  with  how  easily  they  could  now 
stay  in  touch  with  their  child's  work.  One  mother  noted  how  excited  she  was 
because  her  husband's  interest  in  the  computer  had  enticed  him  to  get  involved  in 
their  son's  schoolwork  for  the  first  time. 

Empowerment  at  Home 

The  position  of  the  student  within  the  family  structure  was  considered  by 
many  to  be  enhanced.  For  one  mentally  handicapped  girl,  her  knowledge  of  the 
computer  offered  her  the  chance  to  be  a  "contributing  member  of  the  family  for  the 
first  time."  Another  mother  was  surprised  at  a  mutually  interesting  conversation 
with  her  son  about  her  work  with  databases.  Parents  interviewed  said  that  their 
opinions  of  what  the  children  were  capable  of  achieving  had  increased  dramatically 
since  involvement  in  the  program. 

Problems 

There  were  varied  reports  of  family  tensions  and  anxieties  caused  by  the 
introduction  of  the  computer  into  the  house.  The  extensive  tie  up  of  telephone  lines 
for  electronic  communications  use  was  the  most  highly  reported  problem.  Adult 
use  of  "chat  channel"  for  reasons  ranging  from  extensive  time  on-line  with  friends 
to  flirting  was  also  recognized  as  a  source  of  tension. 

Conclusion 

Buddy  was  successful  in  all  of  the  schools.  The  computers  were 
enthusiastically  received  and  were  being  used  extensively.  There  was  a  high  use  of 
tool-based  software  that  permitted  the  students  to  generate  and  process  information, 
and  a  minimum  use  of  the  computer  for  rewarding  behavior  or  for  rote  instruction. 
Further  there  has  been  a  renewal  effect  on  the  teachers  and  a  positive  impact  on  the 
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culture  of  the  classroom.  In  this  section  we  examine  what  we  feel  are  factors  that 
underlie  the  most  successful  implementation  of  the  program. 

1  flnpth  "f  Tim*  in  thr.  Project 

It  was  clear  from  observations  and  interviews  that  the  first  year  a  school  is 
involved  requires  an  enormous  amount  of  work.  The  logistics  of  getting  die 
cornputer Se  homes  takes  an  incredible  amount  of  plan ning ;  and  energy,  and  no 
aXtof  helpful  hints  takes  the  place  of  actually  going  through  the  ] process.  In 
addition  to  logistical  concerns,  teachers  and  students  are  learning  how  to  use  the 
software  and  what  they  can  do  with  it. 

High  rnmoptp-r  Access 

We  heard  over  and  over  again  that  the  computer  activities  were  only 
possible  bSe  there  were  computers  at  school  and  at  home.  If  ^achej^are  to 
Imaacc  in  larger  student-centered  projects,  it  simply  can  t  all  be  done  at  scnooi. 
T?w  home  computersp^nnit  the  teSiers  to  expect  school  projects  to  continue  at 
home  and  thus  permit  them  to  assign  larger  projects. 

A  Stiirfrnt-Cf  ntftreri  InstmrHonal  Model 

The  computers  are  a  vehicle  for  the  students  to  create  and  manipulate 
information.  This  use  of  the  computers  is  most  successful  in  a  student-centerea 
appS°o  learn  ng,  when  they  £e  used  to  contribute  to  project-based  learning.  In 
SK,  the  teacheS  at  Linden  tended  to  provide  very  detailed ^ruction  to  die 
students  as  to  what  they  will  create,  with  very  few  degrees  of  freedom ^eft  to die 
students  One  of  those  teachers  expressed  a  wish  that  all  the  students  could  have 
£r  own  computer  so  everyone  could  be  doing  the  same  thing  at  the  same  time  aU 
uSr  her  Son.  We  have  doubts  as  to  whether  this  teacher  can  be  success  with 
that  philosophy. 

ft  Purpose,  for  l  l«"g  the  Computers 

If  the  computer  is  to  be  a  tool  for  learning,  then  it  is  essential  that  the  plans 
for  using  the  computers  begin  with  learning  goals.  That  is,  instructional  goals  and 
not  computer  goals  must  drive  the  activity.  However,  the  computers  offer  new 
oppoSfes  for  thinking  about  and  working  in  the  content  areas,  t  is  not  adequate 
to  simply  reflect  on  traditional  learning  goals;  rather  it  is  necessary  to  consider  what 

for  learning  and  understanding  are  available.  There  isnot  such 
SaSPSt  another  tool."  Each  tool  for  learning  offers  particular opportunities 
for  teaming  and  understanding  and  we  must  understand  the  strengths  and  potentials 
so  that  it  is  a  tool  that  is  used  effectively. 

TffaEkr»  Atf"nftd  to  AHnnrinn  "f  Innovations 

It  seems  to  us  that  a  significant  factor  in  successful  implementation  is  the 
presence  of  several  innovators;  teachers  or  administrators  ready  to  take  chances, 
work  at  the  cutting  edge,  and  lead  the  way  for  the  others  at  die  school.  In  addition 
me  success  of  the  prognun  would  not  have  been  possible  without  a  large  amount  of 
extra  effort  on  the  part  of  the  teachers.  All  the  teachers  reported i  spending  long 
hours  learning  about  software  and  trying  to  "keep  up  with  thestudents.  In  all 
case^Se  teachers  reported  that  the  effort  was  well  worth  it.  They  saw  die  value  of 
the  computers,  embraced  the  opportunity,  and  were  willing  to  spend  long  hours. 
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tSSHi VlSLE^'.  ,Ma?y  P/0,eSSl°nalS  WOrkin°  as  '"structional  designers  and/or 

and  Instructional  systems  content  S!« « anJ *  !9  9,  producin9  instructional  materials 
delivery  svstem*  anri  miiif  ,    ?  a"d  emPhasis-  fo™ats  for  delivering  instruction 

SETS' Syr  2ss  r?r  r  ect-  san  F-cisc°  b^  *«  » 

Valley  in  the  companies such  asllJESZ  ?  SUbj6C,S  are  Si9nificant  IT  leaders  in  •»  Silicon 
MiJsystemsT hJ^A^J^  BM'  Hewle"-pa<*ard,  Amdahl,  Sun 

were  distributed.   Sixty-six  V^^^  52  E?  1^ ^ 

neTd^Kstrnlrort^e^ry  l^^1-*  ^P"'  da,a  to  9ain 

.he  reseats  ~Q  J&W^  WhiCh  ™  °<  in"-<  <° 

i  Whfl7c  eh"1S,r1UC,l0nal  d6SiflnerS  have  hj9her  incomes  «han  female  counterparts? 
Whats  the  salary  range  of  the  professionals  who  responded  to  the  ~!vey? 

Summary  of  Section  I  Data:  General  Information. 

individual  survey  pa^llclpanf)  "°  MS  ™n  °iV6n  by  a  particu,ar 

Gender.  38  female  (57.6%)  and  27  males  (40.9%).  One  subject  did  no,  specify  the  gender. 
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Current  position.  Subjects  were  asked  to  mark  all  that  apply. 

•  Instructional  designer/curriculum  developer   42  responses 

•  "Stand-upH  instructor   18  responses 

•  Training  manager   7  responses 

•  Media  and  production  specialist   7  responses 

•  Independent  contractor   13  responses 

•  other   13  responses 

Responses  listed  under  "Other  included  manufacturing  manager,  program/project  manager, 
consultant,  academic  dean,  and  software  engineer. 

There  are  a  total  of  100  responses  from  66  subjects.  This  indicates  that  many  of  the 
instructional  designers/trainers  "wear  two  hatsH. 

Work  setting.  Work  settings  of  survey  participants  are  presented  below. 

•  Business  or  industry  training   42  responses 

•  Health  professions  training   1  response 

•  Agency  (law  enforcement,  military,  etc.)   1  response 

•  College  or  university   12  responses 

•  Other   11  responses 

Responses  under  HOtherH  included  CEO  of  a  training  development  company,  worldwide  video 
conference,  network  manager,  educational  technologist,  leader,  upper  level  management,  high 
school  instructor,  education,  multimedia  software  development,  quality  assurance,  and 
librarian.  Some  participants  checked  more  than  one  work  setting.  Seven  (7)  subjects  did  not 
respond  to  the  work  setting  item. 

Years  in  present  position. 

•  1-3  Years   27  responses  (41.5%) 

•  4-6  Years   12  responses  (18.5%) 

•  7-10  Years    9  responses  (13.8%) 

•  11-15  Years   6  responses  (9.2%) 

•  Over  15  years   11  responses  (16.9%) 

One  (one)1  subject  did  not  respond  to  this  item. 
Salary  level. 

•  $20,000  -  $30,000  Year    3  responses  (4.8%) 

•  $31,000  -  $40,000  Year    11  responses  (17.5%) 

•  $41,000  -  $50,000  Year    13  responses  (20.6%) 

•  $51,000  -  $60,000  Year    15  responses  (23.8%) 

•  Over  $60,000  Year   21  responses  (33.3%) 

Three  (3)  subjects  did  not  respond  to  this  item. 

Degrees  earned.  Subjects  were  asked  to  mark  all  that  apply  and  list  academic  majors. 

•  AA  or  AS:   8  responses 

•  BA  or  BS:   45  responses 

•  MA  or  MS:   47  responses 

•  Doctorate:   13  responses 
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Majors  listed  included  Instructional  Technology,  MBA,  Computer  Science,  Sociology,  Home 
Economics,  Mass  Communications,  Journalism,  Business,  Nursing,  Political  Science,  History, 
Library  Science,  Social  Studies,  Educational  Administration,  Electrical  Engineering,  Educational 
Psychology,  Computer  Engineering,  Mathematics,  Art,  Physics,  Psychology,  Hispanic 
Civilization,  English,  Advertising,  Anthropology,  Math,  Biology,  Zoology,  Theater  Arts,  Liberal 
Studies,  Social  Science,  and  Ancient  and  Modern  languages.  One  subject  had  not  earned  a  post- 
secondary  degree.  Most  frequently  listed  majors  included  Instructional  Technology,  Business, 
Education,  and  Social  Studies. 

Area*  taught  prior  to  the  current  assignment.  Subjects  were  asked  to  mark  all  that 
apply. 

•  K-8  school  classroom   12  responses 

•  9-12  school  classroom   18  responses 

•  Community  College   22  responses 

•  University   18  responses 

•  0,ner   21  responses 

The  listing  under  "Other"  included  non-profit  organization,  professional  association,  private 
language  school,  private  business  school,  private  industry,  seminars,  workshops,  Peace  Corps, 
sales,  software  industry,  adult  education,  Fortune  500  companies,  and  the  military. 

Items  which  were  not  included  on  the  instrument  but  which  may  be  important. 

Items  which  might  be  added  to  the  survey  competencies  found  in  the  instrument  were  listed  by 
participants.  Included  were  public  speaking,  team  skills,  negotiation  skills,  vender 
management  (developing  criteria,  interviewing,  evaluation),  management  expertise,  electronic 
performance  support  systems  and  applications,  knowledge  engineering,  information  mapping, 
hypertutoring,  curriculum  design,  summative  evaluation,  practices  in  business  management, 
crisis  intervention,  technical  writing  skills,  evaluation,  design  and  development  of  simulation 
and  games,  role  playing,  group  dynamics,  consulting  skills,  corporate  training,  writing  skills. 

The  most  frequently  suggested  items  included  those  relating  to  writing  skills,  evaluation,  and 
management  skills. 

Summary  Of  Section  II  Data;  Competencies.  Understanding  Knowledge.  Skills 

Percentages  of  professionals  who  considered  the  competency,  understanding,  knowledge  or 
skill  to  be  important  or  a  high  priority  are  presented  below,  followed  by  the  rank-order  of 
that  item  in  relation  to  other  items  included  in  the  survey.  Percentages  of  respondents  who 
were  undecided,  gave  the  item  a  low  priority,  or  suggested  that  the  item  be  eliminated  from 
consideration  are  also  presented  below. 


722  3 

72  7 


COMPETENCY,  UNDERSTANDING, 
KNOWLEDGE  OR  SKILL 


IMPORTANT  OR  HIGH  PRIORITY 
PERCENTAGE  RANK 


92.4%  to  100%  of  the  respondents  consider  competency 
to  be  important  or  high  priority 

Knowledge,  understanding  and  applications  of 
instructional  design  models  and  principle   

undecided    0.0% 

low  priority  ...  0.0 

eliminate    0.0 


Learning  needs  assessment  and  evaluation; 

understanding,  skills  and  applications    96.9% 

undecided   3.1% 

low  priority  ...  0.0 
eliminate    0.0 


Project  management,  from  inception  to 

completion    93.9% 

undecided    4.5% 

low  priority  ..  1.5 
eliminate    0.0 

Design,  production  and  utilization  of 

self-paced   learning   materials    9  2.4% 

undecided    6.1  % 

low  priority  ...  1.5% 
eliminate    0.0 


81.5%  to  89.4%  of  the  respondents  consider  competency  to  be  important  or 
high  priority 

Instructor-led  training,  including  skills 

necessary  for  giving  effective  presentations    89.4%  5.5 

undecided   7.6% 

low  priority    3 .0 

eliminate    0.0 


Design,  production  and  utilization  of 

Independent  learning   modules    89.4%. 

undecided   7.6% 

low  priority    3.0 

eliminate    0.0 


Learning  theories,  including  adult  learning 

and  cognition    87.9%  7 

undecided   9.1  % 

low  priority   1.5 

eliminate    1  . 5 
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Use  of  computers  in  word  processing,  data 

bases,  and  spread  sheets    86  4   %  8 

undecided   4.5% 

low  priority    6 . 1 

eliminate    3.0 

Computer  based  training  and  computer 

assisted    Instruction    83  3%  g 

undecided   15.2% 

low  priority   1.5 

eliminate   0 . 0 

Project  proposal  writing,  including  all 

essential  elements  needed  for  funding  success    83  1%  10 

undecided    10.8 

low  priority   '..  3.1 

eliminate   3 . 1 

Desk-top  publishing,  including  basic  design, 

layout  and   production    81.5%  11 

undecided   9.2% 

low   priority   9.2 

no  response   1  . 5 

Mgh  'priolftf*  °'       respondents  consider  competency  to  be  Important  or 

Design,  production  and  utilization  of  video 

Instructional    materials   78  %%  2 

undecided   12.1% 

low   priority   9.1 

eliminate   0 . 0 

Evaluation  and  selection  of  "off-the-shelf" 

training  materials ■   78  8% 

undecided    13.6% 

low  priority   7.6 

eliminate   0 . 0 

Design,  production  and  utilization  of  multi- 
media programs,  including  hypermedia    75  8%        *  4 

undecided   19.7% 

low   priority   4.5 

eliminate   0 .0 


Mf^Jrtoritf*  °f       re$P°ndents  eon****'  competency  to  be  Important 


or 


Computer  graphics,  including  basic  design, 

layout   and   production    74  2% 

undecided    10.6% 

low   priority    15.2% 

eliminate    o.O 
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Design,  production  and  utilization  of 

Instructional   Interactive  video   

undecided   15.2% 

low   priority   10.6 

eliminate    0.0 


73.8%  17 


70.3%         1  8 


2  0 


Basic  research  understanding,  skills, 
competencies   

undecided   18.5% 

low  priority   7.7 

eliminate   0.0 

Crosscultural  communication  and  relation- 
ship skills  and  understanding   

undecided   17.2% 

low   priority   10.9 

eliminate   1-6 

60.0%  to  69.7%  of  the  respondents  consider  competency  to  be  important  or 
high  priority 

Time  management,  including  time-line  g 
development  and   applications    69.7% 

undecided    18.2% 

low   priority   9.1 

eliminate   3.0 

Telecommunications,  including  knowledge, 
understanding,  skills  and  applications   

undecided    24.2% 

low    priority   9.1 

eliminate   1  •  5 

Distance  education,  including  administration,  cost 
effectiveness,   technical   requirements    63.6% 

urdecided   2  7.3% 

low  priority   9.1 

eliminate   0.0 

Design,  production  and  utilization  of  displays,  including 

interactive  and   self-instructional  displays    61.5%  22 

undecided   15.4% 

low   priority   20.0 

eliminate    3.1 

60.0%  to  69.7%  of  the  respondents  consider  competency  to  be  important 
or  high  priority 

Administration  and  management  models  and  principles    60.6%        2  3 

undecided   2  7.3% 

low  priority    9.1 

eliminate    3.0 
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c"em  sssssl  manao<"n2e;t.  7to,y  and     *»  «o .  o  %  24 

low  priority    9.2 

eliminate   3.1 

m?XrlV%  01       reSP°ndentS  COnsider  competency  to  be  important  or 

Interpersonal  relationship  theory,  skills  and 
applications   

undecided   .2  4.2% 5  6.1%  25 

low  priority   12.1 

eliminate   7.6 

Financing,  budgeting  and  depreciation    5« 

undecided   30.8%  55.4%        2  6 

low  priority   13.8 

eliminate    q.O 

ST prio5rny%  °'       reSpondents  <»»«*>'  competency  to  be  important  or 

Design,  production  and  utilization  of  overhead 
projection  transparencies 

undecided   18.2% 54.5%  27 

low   priority   2  2.7 

eliminate    4.5 

0rs"TnSerel:re2n: %r and  8ppiiM,i°"s  •«•»«  *• 

low  priority   2 0.0 

eliminate    31 

Futures  studies,  trend  indicator  extrapolations 
development  of  alternative  futures  P°,a"°n8' 

undecided  26.6%  51.6%  29 

low  priority    18.8 

eliminate    3.1 

hig?  prio5rVy%  °'       "V"""*"*'  confer  competency  to  be  important  or 

Design,  production  and  utilization  of  audio 
instructional  materials 

undecided    1  8.2% 51.5%  30.5 

low  priority    28.8 

eliminate    1.5 
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Conflict  resolution  theory,  skills  and 

applications,  and  stress  management    51.5%  31.5 

undecided   22.7% 

low  priority    16.7 

eliminate    9.1 

41.3%  to  44.6%  of  the  respondents  consider  competency  to  be  important  or 
high  priority 

Design,  production  and  utilization  of 

photographic   instructional   materials    44.6%        3  2 

undecided    2  6.2% 

low  priority    26.2 

eliminate    3.1 

Design,  production  and  utilization  of 

flip  charts,  posters  and  other  flat  graphics    41.5%        3  3 

undecided   2  9.2% 

low  priority    21.5 

eliminate   7.7 

Facilities  design  and/or  modification  for  media 

design,   production,   utilization    41,3%        3  4 

undecided   3  6.5% 

low   priority   15.9 

eliminate    6.3 


727  8  vr 


Conclusion*.  A  comparison  of  university  curricula  with  the  data  summary  presented  above 
indicates  that  many  university  curricula  need  to  be  examined  and  revised  in  order  to  meet  the 
needs  of  professional  trainers  and  instructional  designers  in  the  field. 

Most  university  programs  include  many  of  the  "basics"  needed  by  trainers  and  instructional 
designers,  but  have  "gaps"  which  could  be  filled  by  including  courses  in  areas  dealing  with 
human  relationships  and  overseeing  projects  or  programs. 

Facilities  design  and  some  of  the  more  familar  and  comfortable  media  were  not  considered  as 
important  as  courses  or  competencies  in  dealing  with  other  human  beings,  and  management  and 
administration. 

Further  research  is  needed.  The  questionnaire  could  be  revised  to  include  addtional  items 
suggested  by  participants  in  this  study.  More  subjects  and  subjects  in  other  settings  should  be 
surveyed.  Although  many  trainers  work  in  agencies,  only  one  agency  trainer  was  included  in  the 
survey.  Health  professions  trainers  also  need  to  be  surveyed  in  depth. 

Additional  data  have  been  received  since  the  conclusion  of  the  writing  of  this  report  A  more 
comprehensive  analaysis  of  data  will  be  prepared  for  publication  in  the  near  future. 

For  a  copy  of  the  results  or  additional  information  concerning  the  study,  please  contact: 

Dr.  John  E.  Morlan,  Professor  and  Director 
Cooperative  Doctoral  Programs 
College  of  Education 
San  Jose  State  University 
San  Jose,  California  95192 

or 

Dr.  Mei-Yan  Lu,  Assistant  Professor 
Instructional  Technology  Program 
San  Jose  State  University 
San  Jose,  California  95192 

Either  of  us  may  be  reached  at: 

FAX:  (408)  924-3713 
Phone:  (408)  924-3620 
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A  SURVEY  OF  MEDIA  AND  INSTRUCTIONAL  TECHNOLOGY  COMPETENCIES 

NEEDED  BY 

BUSINESS,  INDUSTRY,  HEALTH  PROFESSION,  AGENCY  AND  MILITARY  TRAINERS 

IN  NORTHERN  CALIFORNIA,  USA 
FALL  1992 

DIRECTIONS  FOR  RESPONDING  TO  THE  SURVEY  INSTRUMENT.  Please  respond  by  filling  in  your  response 
on  the  enclosed  Scantron  answer  sheet  next  to  the  number  which  corresponds  with  the  number  of  the 
question  on  the  enclosed  survey  instrument.  SKifc  the  left  side  of  the  Scantron  sheet  where  It  needs 
name  and  social  security  numbers.etc.    All  information  provided  bv  vou  through  vour  individual 
responses  to  the  survey  instrument  will  bo  hftP1  *tri^  confidential.  Please  return  the  questionnaire 
And  Scantron  form  in  the  envelope  provided  for  your  convenience,  or  fax  the  materials  to  us  at 
(408)  924-3713.   If  you  would  like  to  have  a  summary  of  our  findings,  please  write  your  name  and 
address  on  the  back  of  this  instrument  or  call  Mei-Yan  Lu  at  (408)  924-3645. 

PART  I:  GENERAL  INFORMATION 

Mark  all  that  apply  with  a  soft  lead  pencil  on  the  enclosed  answer  form,  for  items  1  -  53. 

1 .  Gender:  (a)  Female 

(b)  Male 

Items  2  -7  describe  your  current  position,  please  mark  the  one(s)  that  best  describe(s)  your  job: 

2.  Position:  Mark  all  that  apply. 

3.  (a)  Instructional  designer/curriculum  developer 

4.  (a)  "Stand-up"  instructor 

5.  (a)  Training  manager 

6.  (a)  Media  and  production  specialist 

7.  (a)  Independent  contractor 

Other:  Please  describe  here  

8.  Work  setting:    (a)  Business  or  industry  training 

(b)  Health  professions  training 

(c)  Agency  (law  enforcement,  military,  etc.) 

(d)  College  or  university 

Other:  Mark  He".  Please  list  

9.  Years  in  present  position: 

(a)  1-3  Years 

(b)  4-6  Years 

(c)  7-10  Years 

(d)  11-  15  Years 

(e)  Over  15  years 

10.  Salary  level:    (a)  $20,000  -  $30,000  Year 


(b)  $31,000  ■  $40,000  Year 

(c)  $41,000  -  $50,000  Year 

(d)  $51,000  -  $60,000  Year 

(e)  Over  $60,000  Year 

Items  11  -  14.  Degrees  earned.  Please  mark  the  degree(s)  you  have: 

11.  (a)  AA  or  AS:  List  degree  major   

12.  (a)  BA  or  BS:  List  degree  major   

13.  (a)  MA  or  MS:  List  degree  major   

14.  (a)  Doctorate:  List  degree  major   

Items  15-19.  Areas  you  have  taught.  Please  mark  all  that  apply: 

15.  (a)  K-8  school  classroom 

16.  (a)  9-12  school  classroom 

17.  (a)  Community  College 

18.  (a)  University 

19.  Other:  Mark  V.  Please  list  
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Section   II:     Instructional  Technology  Professional  Knowledge,  Underatandlno 
and  Competencies  ■■«■■•» 

rH^.«COnSide?a,Ch  °1  the  i,6mS  Which  follow  in  "9^  of  what  V°u  feel  is  important  for  performinq  as 
LofnL        6  h3"'  ^ V0""'6'  ins,ruc,ional  *^°9y  professional.  Do  not  I  m It  you  9 
responses  by  consKlenng  only  what  is  needed  by  you  to  perform  well  in  your  current  position 

Mark  V  if  you  consider  the  item  listed  to  be  essential;  should  be  assigned  a  high  priority 
Ma*  b     you  consider  the  item  listed  to  be  important;  should  not  be  assigned  a  SeX  Lh  priority 
Mark  c  .f  you  are  undecided  as  to  whether  the  item  is  important  or  unimportant  P 
.  you  consider  the  item  to  be  relatively  unimportant;  should  be  assigned  a  low  priority 
.f  you  consider  the  item  to  be  irrelevant;  should  be  eliminated  from  consideration  ' 


Mark  "d 
Mark  "e 


20 

21 

22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 

33. 


fni,nr  i«h  ,  9,3!]d  aPpl,ca,,ons  of  instructional  design  models  and  principles 

uTT       , ra,mn9,  'nClud,ng  Ski"S  necessarv  ,or  9ivi"9  e'^tive  presentations 
Use  of  computers  in  word  processing,  data  bases,  and  spread  sheets 
Computer  graphics,  including  basic  design,  layout  and  production 
Desk-top  publishing,  including  basic  design,  layout  and  production 
Computer  based  training  and  computer  assisted  instruction 
Design,  production  and  utilization  of  instructional  interactive  video 

?SnnmPr0dUC!i0n  3nd  Ul"i2a,i0n  °'  mu,,imedia  Programs  including' hypermedia  programs 
Telecommunications,  including  knowledge,  understanding,  skills  and  applications 
Distance  education,  including  administration,  cost  effectiveness.  techS  requirements 
Design,  production  and  utilization  of  overhead  projection  transparencies  reqU'r8men,S- 
Design,  production  and  utilization  of  audio  instructional  materials 
Design,  production  and  utilization  of  video  instructional  material 
^ntrannder        Uli'iZali°n  *  ph0,09raPhic  instructional  materials,  including 
Design,  production  and  utilization  of  instructional  flip  charts,  posters  and  other  fat  oranhics 
Design  production  and  utilization  of  effective  displays,  including  interactive  9  P 

and  self-instructional  displays. 

Design,  production  and  utilization  of  self-paced  learning  materials 
D«s  9n.  production  and  of  utilization  of  independent  learning  modules 
38.      Evaluation  and  selection  of  "off-the-shelf"  training  materials 

40       LeZn?,  rdS  aSSes1Smem  and  evaluation:  understanding,  skills  and  applications. 
4  0.       Learning  theories,  including  adult  learning  and  cognition 

Conflict  resolution  theory,  skills  and  applications,  and  stress  management 
Interpersonal  relationship  theory,  skills  and  applications 
Administration  and  management  models  and  principles 
Time  management,  including  time-line  development  and  applications 
Client  centered  management  theory  and  implementation 
Organizational  development  theory  and  applications 

Project  proposal  writing,  including  all  essential  elements  needed  for  funding  success 
Program  management,  from  inception  to  completion.  9 
49.      Finance,  budgeting  and  depreciation. 
«n       Basic  research  understanding,  skills,  and  competencies 

Fu3'l1U,rH  C0TUnJCa,i0n  3nd  rela,ionshiP  ^ills  and  understanding. 
Futures  studies,  trend  indicator  extrapolations,  development  of  alternative  futures 
Fachties  design  and/or  modification  for  media  design.production.  2 Son 

Please  add  items  you  think  are  important  that  are  missing  from  the  preceding  list  in  the  space  below. 
Thank  you  very  much  for  your  time,  and  sharing  your  prof.a.lona.  judgement  with 


34. 
35. 

36. 
37. 


41 
42 
43 
44 
45 
46 
47 
48 


50 
51 
52 
53 
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The  Critical  Role  of  the  ID  in  Interactive  Television: 
The  Value  of  Immediacy 


Introduction 

Interactive  television,  or  video  conferencing,  includes  compressed  digital  video 
conferencing,  ITFS,  and  microwave  television.  Video  conferencing  has 
opened  new  avenues  for  delivering  high  quality  instruction  to  greater  numbers 
of  students  in  higher  education.  This  is  true  at  the  University  of  Wyoming  and 
in  several  other  states  where  video  conferencing  has  been  instituted  statewide 
in  higher  education.  With  the  potential  for  offering  high  quality  instruction  to 
large  numbers  of  higher  education  students,  however,  has  come  challenges  to 
the  administration,  to  instructors,  and  even  to  the  students.  One  of  these 
challenges  is  to  minimize  barriers  to  learning  at  a  distance,  as  distant  students 
tend  to  feel  isolated  and  left  out.  Whittington's  (1987)  research  shows  that  no 
significant  difference  is  found  between  learning  outcomes  in  face-to-face 
instruction  and  in  instruction  that  is  mediated  by  technology.  However,  wide 
gaps  do  exist  in  student  attitudes  toward  learning,  some  of  which  have  been 
shown  to  influence  student  achievement. 

While  it  may  be  possible  for  campus-based  instructors  to  get  away  with 
lecturing  for  three  hours  each  week  in  class,  this  kind  of  approach  is  totally 
inappropriate  for  instructors  teaching  via  video  conferencing.  Can  you 
imagine  trying  to  learn  by  watching  a  television  monitor  from  across  a  large 
room  for  such  extended  periods  of  time?  The  challenge  to  the  distance 
instructor  becomes  even  greater  when  the  instruction  is  primarily  lecture,  with 
some  charts,  graphs,  and  text  to  "liven  up"  the  lesson.  It  is  particularly 
important  for  distance  instructors  to  incorporate  behaviors  in  their  teaching  that 
will  reduce  the  learners'  sense  of  physical  and  psychological  distance. 

One  way  to  reduce  this  sense  of  distance  is  for  instructors  to  use  immediacy 
behaviors,  or  communication  behaviors  that  "enhance  closeness  to  and 
nonverbal  interaction  with  another"  (Mehrabian,  1969).  Immediacy  behaviors 
convey  approachability  and  communicate  interpersonal  warmth  and  closeness. 
Typical  immediacy  behaviors  include  the  use  of  a  variety  of  vocal  expressions 
when  teaching,  having  a  relaxed  body  position  when  talking  to  the  class,  and 
smiling  at  the  class  as  a  whole.  The  "social  presence,"  or  the  ability  to 
approximate  the  characteristics  of  face-to-face  interaction,  is  obviously  limited 
in  mediated  instruction.  Correlations  have  been  found  between  immediacy  and 
student  expectancies  (Giglio  &  Lustig,  1987)  and  between  immediacy  and 
student  motivation  (Christophel,  1990),  which  in  turn  result  in  higher 
achievement.  Thus,  instructors  who  employ  immediacy  strategies  to  increase 
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perceived  social  presence  are  likely  to  enhance  both  student  learning  and 
satisfaction  in  video  conferencing  classes. 

A  key  figure  in  increasing  immediacy  behaviors  among  distance  teaching 
faculty  is  the  instructional  designer.  This  individual  plays  a  critical  role  in  the 
development  and  continuing  viability  of  each  course  taught  at  a  distance.  The 
role  of  the  instructional  designer  has  three  components:  in  relationship  to  the 
infrastructure,  in  relationship  to  the  technology,  and  in  relationship  to  people- 
specifically,  students  and  faculty.  First  we  will  discuss  the  relevance  of 
immediacy  behaviors  to  distance  education,  particularly  in  video  conferencing 
environments.  Following  is  a  discussion  of  ways  that  the  instructional 
designer  can  help  foster  immediacy  teaching  behaviors  through  the 
infrastructure,  the  technology,  and  the  people  involved  in  video  conferencing. 

While  the  focus  is  on  interactive  video  conferencing,  similar  strategies  can  be 
used  to  achieve  similar  results  with  less  interactive  distance  teaching 
technologies.  These  "non-interactive"  technologies  include  one-way  video 
transmission,  either  through  satellite  teleconferencing  or  broadcast  or  cab'«s 
television,  or  even  videotapes.  All  of  these  technologies  are  usually 
supplemented  with  printed  materials  which  are  sent  to  students  at  their  distance 
learning  locations. 


Immediacy  in  the  Context  of  Distance  Education 

Compelling  evidence  exists  in  traditional  face-to-face  learning  contexts  that 
decreased  physical  and/or  psychological  distance  between  teachers  and  students 
is  associated  with  enhanced  learning  outcomes.  Lawrence  Rifkind  (1992) 
summarized  research  on  immediacy  as  a  predictor  of  teacher  effectiveness  and 
learner  achievement  in  the  interactive  television  classroom.  Studies  primarily 
in  the  area  of  communication  have  found  positive  correlations  between 
immediacy  and  cognitive  achievement  as  well  as  between  interactive  television 
system  design  and  cognitive  achievement  (e.g.,  Christophel,  1990;  Giglio  & 
Lustig,  1987;  Gorham,  1988). 

Mediated  learning  is  most  effective  when  students  perceive  a  personal 
involvement  in  the  educational  process.  Helping  distant  students  become 
personally  involved  in  their  mediated  education  is  achieved  through  "guided 
didactic  conversation"  (Holmberg,  1984).  Holmberg  describes  internalized 
conversation  as  occurring  through  interaction  with  course  material,  and 
externalized  interaction  as  conducted  directly  with  the  instructor  verbally  by 
telephone  or  other  telecommunication  means.  In  addition  to  these  two  forms 
of  interaction,  it  is  generally  understood  that  a  third  type  of  interaction  is 
conducive  to  learning  and  is  particularly  important  in  distance  education: 
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interaction  among  the  learners  themselves  (Moore,  1989).  University  of 
Wyoming  course  evaluations  indicate  that  when  students  interact  with  other 
students  in  the  class,  they  find  the  class  both  intellectually  stimulating  and 
personally  relevant  (Farr,  1992).  While  the  instructor  may  not  play  a  strong 
role  m  this  land  of  interaction,  the  sense  of  immediacy  is  clearly  present 
within  the  local  groups.  H 

How  can  distance  faculty  guide  their  students'  didactic  conversation'' 
Unfortunately  faculty  members  designated  as  distance  education  instructors 
typically  lack  the  knowledge  and  expertise  to  adapt  their  courses  successfully 
to  distance  delivery  on  their  own  (Farr,  Murphy  &  Flatt,  1992).  Therefore, 

ronint  I •  •  ^  neCdS  t0  occur'  Faculty  development  should  focus  on  me 
concept  of  immediacy  to  engage  students  in  didactic  conversation.  Students 
who  engage  in  didactic  conversation  (with  the  instructor,  the  content,  or  each 

afm^r^sfectoT      **  inV°1Vemem  "  thC  leaming  prOCCSS 

Immediacy  and  the  Infrastructure 

Instructional  designers  are  a  vital  part  of  the  infrastructure  that  provides  the 
interactive  television  system.  These  individuals  should  participate  in  the 
decision-making  that  will  impact  policy  statewide.  Theyare  more  likely  to 
have  firsthand  toowledge  of  the  factors  that  influence  scheduling,  fac2 
needs  new  product  development,  and  technological  problems.  With  this 
knowledge  base,  instructional  designers  can  assist  distance  education 

™d  PraCtidn8  the        °f  teachin*  that 

^J?m}nl  uC  inf?structure  incl"des  the  following:  the  seven  community 
colleges  (which  together  with  the  University  of  Wyoming  comprise  to  current 

^leC°rUnCaUOnS  "etW°rk);  Six  academic  coordinators  who  arc  iS 
at  community  colleges;  colleges  and  departments  at  the  University  of 
Wyoming;  other  users  on  campus;  the  state  telecommunications  system;  the 
telecommunications  service  provider;  and  the  vendor. 

At  the  University  of  Wyoming,  the  instructional  designers  were  involved 
tengentially  in  the  decision-making  regarding  the  purchase  of  to  compressed 
digitel  video  conference  equipment.  Since  the  purchase  and  ins^htion  0f  to 
equipment,  however,  the  instructional  designers  have  been  involvedriUerallv 
wthT*«  h  ^  infraftructure-  ™»  instructional  designer  maintain  coS " 
1™   S  MdKtS  at  *"  SlteS'  f,rst  through  a  studen*  orientation  and 
evSZ^  ^pmon;tonn8  individual  sessions  and  conducting  formative 
evaluations.  These  forms  of  contact  are  particularly  pertinent  in  promoting 
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immediacy  behaviors  of  students  whether  they  meet  face-to-face  with  their 
instructor  or  learn  at  a  distance.  The  responsibilities  of  the  instructional 
designer  associated  with  the  infrastructure  include: 

*  providing  a  student  orientation  and  monitoring  classes; 

*  conducting  live  formative  evaluations  in  courses; 

*  spearheading  weekly  conference  calls  with  the  State  Division  of 

Telecommunications  and  US  West; 

*  arranging  and  conducting  workshops  for  academic  coordinators, 

instructors,  and  administrators  on  the  use  of  the  system; 

*  maintaining  ongoing  contact  with  the  far-flung  academic  coordinators 

and  the  site  facilitators,  and  on-campus  producer/directors  and 
engineers; 

*  monitoring  and  integrating  audio,  video,  and  computer-based 

technologies. 


Immediacy  and  the  Technology 

The  technology  of  compressed  digital  video  conferencing  is  fairly  recent  in 
higher  education,  although  business  and  industry  have  been  using  the 
technology  for  over  a  decade.  The  University  of  Wyoming  has  recently 
implemented  a  statewide  compressed  video  conferencing  system.  What  are  the 
critical  attributes  of  video  conferencing? 

*  Video  conferencing  provides  two-way  interactive  video  and  audio 
transmission  among  two  or  more  sites  simultaneously.  That  means  that 
students  in  widespread  locations  can  see  and  hear  their  instructor  as 
well  as  each  other  almost  as  if  they  were  in  the  same  room. 

*  Video  conferencing  also  enables  live  transmission  and  annotation  of 
text  and  graphics.  Instructors  and  students  can  use  a  hand-held  control 
to  activate  a  special  camera  that  transmits  images  and  graphics  to  the 
other  sites.  Then  they  can  annotate  these  "slides"  from  their  own 
locations  using  a  graphics  pad  and  pen. 

*  The  same  hand-held  remote  control  can  be  used  to  zoom  in  on  the 
face  of  a  speaker. 

*  Peripherals  enable  films  and  videotapes  to  be  shown  in  real  time  over 
the  system. 

*  Because  video  conferencing  is  PC-based,  the  system  allows  the 
demonstration  of  computer  software  in  a  point-to-point  conference  by 
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means  of  telecommunications  programs  such  as  PCAnywhere.  In 
addition,  instructors  can  show  slides  of  the  software  program  to  all 
students  in  multi-point  conferences. 

*  Video  conferencing  digitizes  video  and  sound,  which  are  sent  over 
telephone  lines  to  other  locations,  where  they  are  converted  back  to 
images  and  sound.  Because  the  entire  picture  is  not  being  transmitted 
continually  as  in  the  case  of  broadcast  video,  the  images  and  sound 
seem  to  be  slightly  distorted. 

How  can  these  attributes  be  used  to  foster  immediacy  in  distance 
teaching/learning  contexts?  Instructional  designers  can  assist  faculty,  students, 
and  even  producer/directors  and  camera  operators  in  exploring  and  capitalizing 
on  the  attributes  of  video  conferencing. 

The  University  of  Wyoming  instructional  design  procedure  connected  with  the 
technology  follows:  The  instructional  designer  him/herself  must  first  learn  to 
use  the  technology.  Then  the  instructional  designer  must  figure  out  teaching 
strategies  that  exploit  these  attributes.  Next  the  instructional  designer  works 
with  individual  instructors  on  their  needs,  objectives,  and  teaching  strategies. 
Only  then  can  the  instructional  designer  help  an  instructor  select  and  practice 
immediacy  behaviors  with  the  technology. 

Specific  ways  that  the  instructional  designer  can  exploit  the  attributes  of  video 
conferencing  to  foster  immediacy  behaviors  include: 

*  Teach  instructors  and  students  to  use  the  equipment  so  that  they  can 
send  and  receive  slides  and  graphics,  and  use  the  graphics  tablet  for 
annotating  graphics. 

*  Train  site  facilitators,  or  monitors  at  each  receiving  location  to  assist 
the  instructor  in  the  use  of  the  technology. 

*  Work  with  a  camera  operator,  the  instructor,  or  a  facilitator  who 
does  the  camera  work  at  the  teaching  site  to  pan  around  or  zoom  in  for 
extreme  close-up  of  the  face  of  the  instructor. 

*  Work  with  a  producer/director  in  a  television  studio  to  use  visual 
effects  to  show  the  instructor's  graphics  and  the  face  of  the  instructor 
simultaneously.  These  effects  avoid  having  students  at  receiving 
locations  view  only  the  graphics  for  extended  periods  of  time. 

*  Ensure  that  students  are  seated  so  that  they  can  view  the  monitors 
comfortably.  Students  should  also  have  easy  access  to  microphones  and 

6 


ERIC 


736 


be  trained  in  advance  in  the  use  of  the  microphone"  co  avoid 
"necrophobia." 

*  Ensure  that  instructors  use  hands-free  microphones  so  that  they  can 
devote  their  attention  to  teaching,  interacting  with  students,  and  using 
the  equipment  effectively. 


Immediacy  and  the  People 

Students 

At  the  beginning  or  even  prior  to  the  first  class  session,  the  instructional 
designer  conducts  an  orientation  to  learning  via  video  conferencing.  This 
orientation  may  include  a  training  videotape.  Orientation  sessions  should 
enable  students  to  practice  using  the  video  and  audio  equipment,  including 
microphones,  and  ideally  should  provide  guidelines  to  success  in  learning  via 
video  conferencing. 

The  instructional  designer  must  ensure  that  site  facilitators  are  well  trained  in 
the  use  of  video  conference  equipment.  At  a  receiving  location,  the  site 
facilitator,  whose  primary  responsibility  is  to  assist  the  instructor,  can  also  act 
as  an  advocate  of  the  students.  During  class  sessions  these  facilitators  can 
inform  the  instructor  when  shy  students  want  to  participate  but  may  be 
reluctant  to  speak  up  cr  use  the  microphone.  Outside  of  class  they  can  help 
students  desigi.  graphics  that  are  appropriate  for  the  medium  and  provide 
practice  with  the  graphics  pad  and  pen  as  the  students  prepare  for  their 
presentations. 

Faculty 

Faculty  members  often  need  to  be  convinced  to  teach  via  video  conference. 
Once  convinced,  they  need  to  learn  to  alter  their  teaching  methods  to  be  more 
appropriate  for  teaching  via  video  conference.  Distance  teaching  faculty  have 
been  found  to  be  preoccupied  with  determining  the  course  content  and 
matching  the  content  to  the  time  available,  with  little  attention  paid  to  the 
delivery  of  instruction  and  the  intended  learning  outcomes  (Wolcott,  1991). 
As  described  earlier,  the  core  of  faculty  development  should  be  immediacy 
behaviors  to  engage  students  in  didactic  conversation.  Faculty  development 
involves  identification  of  ways  to  exploit  the  attributes  of  the  technology  with 
the  intention  of  creating  a  positive  learning  environment  for  all  distant 
students.  Instructional  designers  should  help  faculty  focus  on  the  three  modes 
of  interaction:  instructor-students,  student-student,  and  content- student 
(Moore,  1989). 
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Following  is  a  description  of  the  faculty  development  process  at  the  University 
of  Wyoming.  This  process  helps  the  faculty  look  at  the  distance  learner  as  the 
nucleus.  The  instructor  receives  assistance  from  the  instructional  designer 
through  workshops  and  individual  sessions,  from  the  office  through  its 
coordination,  and  from  evaluations. 

Workshops  -  Faculty  development  workshops  occur  in  two  parts.  The 
first  part  focuses  on  general  information  about  distance  education,  including 
information  about  the  ways  that  the  system  is  coordinated  at  the  university  and 
throughout  the  state.  In  this  part  the  instructional  designer  doesn't  teach 
specifically  about  any  given  distance  education  technology.  Instead,  the 
faculty  are  introduced  to  the  philosophy  of  distance  education,  they  gain  an 
overview  of  all  delivery  systems,  and  they  learn  what  is  required  of  them  to 
prepare  for  their  course,  which  they  will  teach  the  following  semester.  This 
segment  is  conducted  by  video  conference,  so  that  faculty  who  will  teach  from 
other  sites  can  participate  as  well.  Thus  the  faculty  are  introduced  to  a 
technology  by  using  it,  and  they  observe  appropriate  teaching  strategies  and 
behaviors  that  the  instructional  designer  models  for  them. 

The  followup  part  of  faculty  development  workshops  focuses  on  applications  to 
specific  delivery  systems.  Instructors  planning  to  teach  via  video  conference 
participate  in  a  video  conference  workshop.  The  focus  of  this  workshop  is  on 
the  course  presentation  by  that  particular  delivery  system.  The  faculty  gain 
expertise  in  using  the  technology  that  they  will  use  in  teaching  while 
demonstrating  and  receiving  feedback  on  a  specific  teaching  strategy. 
Experienced  teleconferencing  faculty  join  inexperienced  faculty  in  these 
workshops  to  share  ideas  and  expertise. 

Individual  sessions  -  These  sessions  with  faculty  begin  with  what  is 
termed  a  pre-production  planning  meeting.  Topics  of  discussion  begin  with 
the  instructor's  teaching  or  learning  objectives.  They  include  teaching 
techniques,  any  special  demonstrations  or  projects,  and  media  presentation 
issues.  Instructors  bring  to  this  meeting  the  following:  the  course  syllabus  an 
outline  of  the  first  session,  and  samples  of  graphics  and  other  visual  aids  that 
they  plan  to  use.  A  followup  individual  session  usually  consists  of  a  practice 
video  taping  session  that  takes  place  in  the  television  classroom.  For 
instructors  who  will  teach  via  video  conferencing,  this  session  simulates  a 
video  conference,  so  that  the  instructor  can  view  their  videotape  in 
compression  format.  It  is  imperative  in  this  session  that  faculty  evaluate  their 
videotapes  for  the  presence  or  absence  of  immediacy  behaviors.  For  example 
they  observe  whether  they  maintain  eye  contact  with  the  camera,  or  how  they ' 
encourage  or  discourage  student  participation. 
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Office  coordination  -  The  Office  of  Off-campus  Credit  Courses,  in  the 
School  of  Extended  Studies  and  Public  Service,  provides  coordination  and 
assistance  to  each  faculty  member.  These  services  include  scheduling, 
providing  course  sites  with  all  the  necessary  course  information  and  supplies, 
publicity,  registration,  and  media  personnel.  While  the  significance  of  office 
procedures  may  not  be  readily  apparent,  a  timely  response  to  instructors'  and 
students'  requests  is  imperative  to  the  smooth  functioning  of  a  course. 

Evaluations  -  Evaluations  dccur  in  each  course.  The  instructional 
designer  conducts  live  formative  evaluations  approximately  one-third  of  the 
way  through  the  course.  The  purpose  of  the  formative  evaluations  is  to  detect 
any  problems  associated  with  the  delivery  of  the  course,  so  that  the  problems 
can  be  addressed.  In  video  conference  classes  the  formative  evaluation  is 
conducted  over  the  video  conference  system.  Instructors  receive  a  typed 
transcription  of  their  students'  discussion.  These  evaluations  provide  the 
students  with  an  opportunity  to  look  critically  at  the  course  as  a  whole  and 
offer  relevant  suggestions.  At  the  University  of  Wyoming  the  instructional 
designer  asks  these  three  questions: 

1.  What  do  you  like  about  the  course  so  far  this  semester? 

2.  What  don't  you  like  about  the  course  so  far  this  semester? 

3.  How  would  you  recommend  that  changes  be  made? 

Summative  evaluations  are  administered  in  written  form  at  the  end  of  the 
course.  Students  address  demographic  issues  followed  by  questions  related  to 
aspects  of  learning,  the  technology,  interaction,  satisfaction,  and  future 
recommendations  about  such  courses. 

In  general,  an  instructional  designer  at  the  University  of  Wyoming  monitors 
the  progress  of  a  course  by  remaining  in  contact  with  the  producer/directors, 
academic  coordinators,  site  facilitators,  students,  and  certainly  the  instructors. 
Instructional  designers  at  the  University  of  Wyoming  begin  with  the 
assumption  that  everyone  wants  to  be  a  good  teacher,  with  the  result  that 
distance  teaching  faculty  typically  improve  their  teaching  and  teaching 
evaluations.  Standard  faculty  development  strategies  at  the  University  of 
Wyoming  (Fair,  Murphy,  &  Flatt,  1992)  include  the  following: 

*  Videotape  practice  sessions 

*  Encourage  mentoring 

*  Brainstorm  ideas 

*  Accentuate  the  positive 

*  Enable  faculty  to  work  with  groups  from  the  same  discipline 

*  Help  faculty  think  from  students'  perspective 

*  Provide  "resources"  not  available  on  campus 
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*  Save  the  instructor's  time 

*  Offer  opportunities  for  research 

*  Conduct  formative  as  well  as  summative  evaluations 


Summary 

Faculty  development  provides  the  opportunity  for  distance  education 
instructors  to  learn  to  exploit  the  attributes  of  video  conference  technologies 
and  to  practice  using  the  technology.  From  instructional  designers  the 
instructors  also  learn  ways  to  incorporate  communication  behaviors  in  their 
teaching  that  will  increase  social  presence  while  reducing  the  learners'  sense  of 
physical  and  psychological  distance.  Some  immediacy  teaching  behaviors  that 
have  been  used  effectively  in  video  conferencing  contexts  follow. 

*  Provide  telephone  office  hours 

*  Provide  written  comments  to  students  on  assignments/exams 

*  Use  personal  examples 

*  Use  "weM 

*  Smile 

*  Maintain  relaxed  body  posture 

*  Use  hand  gestures 

*  Lean  toward  camera 

*  Maintain  eye  contact  with  the  camera 

*  Use  a  variety  of  vocal  expressions 

*  Use  humor 

*  Encourage  participation 

*  Ask  questions 

*  Ask  students  their  feelings  about  assignments,  deadlines,  or  discussion 

topics 


10 


740 

o 

ERLC 


REFERENCES 


Christophel,  D.M.  (1990).  The  relationship  among  teacher  immediacy 
behaviors,  student  motivation,  and  learning.  Communication 
Education,  39(A),  323-340. 

Farr,  C.W.  (1992).  Spring  1992  student  evaluations  of  teleconferencing 
courses.  (Unpublished  manuscript).  Laramie,  WY:  University  of 
Wyoming,  School  of  Extended  Studies  and  Public  Service. 

Farr,  C.W.,  Murphy,  K.L.,  &  Flatt,  L.W.  (1992).  Overcoming  inherent 

obstacles:  How  to  convince  recalcitrant  faculty  to  do  what's  best.  In 

C.  Gibson  (Ed.),  From  vision  to  reality:  Providing  cost-effective, 
quality  distance  education,  Eighth  Annual  Conference  on  Distance 
Teaching  and  Learning,  pp.  45-49.  Madison,  WI:  University  of 
Wisconsin-Extension . 

/ 

Giglio,  K.,  &  Lustig,  M.W.  (1987).  Teacher  immediacy  and  student 

expectations  as  predictors  of  learning.  Paper  presented  at  the  Annual 
Meeting  of  the  Western  Speech  Communication  Association,  Salt  Lake 
City,  UT. 

Holmberg,  B.  (1984).  Guided  didactic  conversation  in  distance  education.  In 

D.  Sewart,  D.  Keegan,  &  B.  Holmberg  (Eds.),  Distance  education: 
International  perspectives,  pp.  114-122.  London:  Routledge. 

Mehrabian,  A.  (1969).  Some  referents  and  measures  of  nonverbal  behavior. 
Behavioral  Research  Methods  and  Instruments,  1,  213-217. 

Moore,  M.  (1989).  Editorial:  Three  types  of  interaction.  The  American 
Journal  of  Distance  Education,  3(2),  1-6. 

Rifkind,  L.J.  (1992).  Immediacy  as  a  predictor  of  teacher  effectiveness  in  the 
instructional  television  classroom.  Journal  of  Interactive  Television 
(ITV),  1(\),  31-40. 

Whittington,  N.  (1987).  Is  instructional  television  educationally  effective?  A 
research  review.  The  American  Journal  of  Distance  Education,  1(1), 
47-57. 


11 


711 

ERIC 


Title: 

Criticizing  Instructional  Materials  Evaluation: 
Adding  Meaningful  Dimension 


Authors: 

Randall  G.  Nichols 
Rhonda  &  Robinson 
BethWiegmann 


r  1  •  ■« 

i'-i  t 

743 


Criticizing  Instructional  Materials  Evaluation: 
Adding  Meaningful  Dimension 


Overview 


nf  in*   ?ti(f  CValuation  of  instn,cti0Ml  trials  is  an  important  but  often  neglected  aspect 

iSTSSSSS-  ^  7°  °0t  -tn.ction7S.Ts7n 
sT^d  S  ,  '  gCnen,Iized'  ^h-based  prescriptions  for  doing 

so.  And  only  rarely  do  teachers  consider  criteria  beyond  those  which  am  uAievemmf  .Ji 
toch^y  oriented  (Rothe,  1991).  Instead,  we  are  Ukely 

«tuated  routines  to  select  and  evaluate  materia.  This  study  suggests  evaluTon  LurTre 
Really  grounded  m  socio^ultural  and  local,  context-sensitive  b««es  iCSZ^c^S 
s^on/eviduauon  guides.  It  reviews  the  development  of  ev^uation^l"  pTidLTw 
measures  and  applies  new  measures  to  current  visual  materials.  A  new  evaluation  i„s,n,^ 
was  developed  and  applied  to  materials  for  teacher  eduction.  f£££^^5T£ 

stereo^  T*  10  rcVCal  *  DUmbcr  0f  e*PIicit  «d  33  So2c2 

n^ulrT^  *°*  °Ur  ,b0Ut  te»chcre-  tcacmng^lassroo^d 

S!™if^  ,  b?eve  "  °Ur  pr°ject  contmucs  exMiiiiiflg  this  visual  material 
will  prov.de  new  understanding  of  our  field  and  our  educational  beliefs 


Background  and  Rationale 


svfitemfltipVi!ati°n  f-  m"terialS  ^  tn,diti0M"y  ^  derived  from  behavioral  theory  and  the 
systematic  design  of  instruction  so  basic  to  our  field.   Within  these  theories  ZriL  is 

SSioSJh-  rCPtftC?ly  P0"1^  out  ^  current  critics  of  educational  technology 
Bowers  (1988)  discusses  the  "current  blindness"  of  the  "conduit"  view  of  software  Z^u.Zl' 

.  ,  ^e  fieId  of  educational  communications  needs  to  revisit  our  theoretical  basis  for  .h. 
simple  models  of  evaluation  or  selection  which  we  have  been  using  orTC  Ken  1985 

iimiuU^iXd^r^r^'90'  ?t  r  ,,,M|"><,'•  "d "■»-»  °°  *• 
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we  wish  to  explore.  Those  with  different  maps  tend  to  take  different  roads."  (p.  9)  Eisner 
points  out  that  our  methods  create  filters  on  our  perceptions. 

Yeaman  (1991)  discusses  the  importance  of  the  socio-cultural  aspects  of  our  field, 
which  he  calls  neglected.  "Educational  communications  and  technology  research  concentrates 
on  the  efficiency  of  learning  stimuli,  instructional  techniques  and  matching  these  with  student 
characteristics."  (p.  990) 

And  Ellsworth  (Ellsworth  and  Whatley,  1990)  has  commented  that:  "Such  research 
privileges  questions  of  how  individuals  process  information  from  educational  media  over  and 
above  questions  about  the  ideological  work  of  meaning  construction,  content  selection,  setting 
of  objectives,  and  media  use.  The  concept  of  information  processing  as  developed  within 
cognitive  psychology  and  applied  in  educational  technology  cannot  explain  meaning,  intent  and 
significance."  (p.  49) 

An  examination  of  evaluation  from  a  socio-cultural  viewpoint  begins  to  provide  more 
powerful  and  meaningful  dimensions  in  the  evaluation  of  media  materials,  historical  or  cutting 
edge.  This  project  begins  to  integrate  media  criticism  and  post-structuralism  as  a  basis  for 
improved  evaluation,  and  provides  a  sample  application.  The  application  involves  visual 
materials  for  teacher  education:  film,  video  and  multi-media. 

Teachers  have  conceptual  and  attitudinal  knowledge  of  technology  based  on  their 
experience  and  the  methods  and  materials  used  to  teach  them  about  the  technology.  These 
materials  remain  largely  unexplored  from  both  an  aesthetic  and  a  critical  or  formal  perspective. 
Likewise,  only  recently  have  a  few  scholars  turned  their  critical  attention  to  educational  films 
and  videos  in  order  to  explain  their  messages,  overt  and  otherwise.  The  visual  representations 
and  the  actual  construction  of  knowledge  in  these  materials  are  now  beginning  to  be 
investigated.  Ellsworth's  (Ellsworth  and  Whatley,  1990)  research  is  an  example  of  such  work 
and  is  used  to  guide  the  work  presented  here. 

In  their  text,  The  Ideology  of  Images  in  Educational  Media,  (1990),  Ellsworth  and 
Whatley  discuss  this  recent  move  to  examine  the  ideology  of  images.  They  point  out  that  until 
very  recently,  educational  communications  research  was  still  focused  on  effectiveness  and  on 
how  learners  gain  from  mediated  messages.  Recently,  educational  media  researchers  have 
turned  their  attention  to  educational  films  and  other  materials,  and  are  investigating  the  visual 
representations  and  the  actual  construction  of  knowledge.  Ellsworth's  study  cites  examples  of 
research  examining  science  and  health  films  and  print  materials. 

The  aim  of  this  multi-phase  project  is  to  suggest  substantial  changes  in  the  ways 
teachers  evaluate  curricular  materials  for  teaching;  to  begin  an  historical  record  of  our  field 
in  terms  of  film  and  video;  and  to  bring  a  more  grounded  and  broadly  literate  understanding 
of  instructional  materials  to  both  theorists  and  practitioners. 

The  results  will  be  summarized  and  examined  for  the  messages  we  can  discover 
involving  materials,  teachers,  pedagogy,  and  technology.  In  addition,  this  work  will  focus  on 
the  hidden  curriculum  of  such  messages,  describe  what  was  considered  valid  knowledge,  and 
detennine  who  possessed  it.  Our  educational  culture  will  be  described,  and  new  ways  of 
looking  at  educational  media  will  be  explored.  In  completing  this  project  we  will  produce  a 
brief  history  of  the  teaching  of  media  utilization  and  evaluation,  test  a  new  evaluation  tool,  and 
analyze  several  mediated  materials. 
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Analyzing  Materials 


The  presentation  itself  was  an  introduction  to  postmodernism  and  its  emergence  as  an 
important  viewpoint  for  educators  (see  Suggested  Reading  List),  followed  by  an  evaluation 
activity.  Questions  raised  by  a  postmodern  perspective  include  issues  of  emancipation,  ethical 
behaviors,  and  empowerment.  In  media  postmodernism  leads  us  to  question  who  is 
delegitimked  and  "othered"  by  mass  media  such  as  television.  These  same  questions  were 
then  raised  while  reviewing  a  newer  educational  videotape  from  a  series.  A  new  evaluation 
form  based  upon  work  by  Rothe  (1991)  was  shared.  This  form  was  prepared  using  socio- 
cultural  analysis  aspects  to  provide  several  meaningful  new  dimensions  to  the  evaluation 
process  (see  Instructional  Materials  Evaluation).  Participants  were  invited  to  use  and  critique 
this  new  instrument,  and  comment  upon  its  perspective  and  its  usability.  Discussion  followed 
about  the  process  and  the  viability  of  socio-cultural  evaluation  in  schools  and  by  teachers.  The 
new  form  encourages  viewers  to  "see"  in  a  new  way;  to  not  ignore  important,  if  subliminal, 
visual  aspects. 

This  presentation  session  suggested  that  typical  evaluation  forms  and  models  are  little 
used  because  they  are  severely  restricted  in  their  conceptions  of  learning.  Learning  is 
conceived  narrowly  as  the  learning  objectives  and  the  methods  used  to  get  students  to  achieve 
those  objectives.  Too  often,  evaluation  of  instruction  and  instructional  materials  does  do  not 
fully  consider  the  political,  economic,  language,  contextual,  and  other  cultural  influences  on 
learning. 

Take,  for  example,  the  idea  of  rule  learning.  A  conventional  approach  usually  asks 
us  to  evaluate  materials  based  on  whether  or  not  students  learned  the  rules  of,  say,  written 
English  as  specified  by  learning  objectives.  We  want  to  say  that  someone  has  learned 
something  more  important  when  he  or  she  has  learned  those  rules  and,  additionally, 

understands  a  variety  of  meanings  of  using  the  formal  rules.  A  person  should  k  *hen,  in 

what  context,  she  has  the  option  to  use  one  set  of  languages  rules  or  another. 

These  extended  understandings  of  knowing/learning  are  crucial  because  they  are  in 
realms  where  many,  even  most,  of  the  important  influences  on  learning  reside.  We  can  select 
a  material  based  upon  what  it  explicitly  teaches  and  how  it  explicitly  teaches  discrete 
knowledge,  but  if  that  learning  is  politically,  ecologically,  or  economically  discouraging  to  the 
learner  and  the  learner  doesn't  get  a  chance  to  understand  the  politics,  ecology,  or  economics, 
to  that  extent  learning  (and  teaching?)  is  a  failure.  For  example,  if  learning  is  taught  as  a 
predominantly  individual  and  cognitive  activity  devoid  of  any  emotion  or  community,  other 
useful  ways  of  knowing  are  apt  to  be  stunted  or  dominated  for  powerful  purposes. 

Likewise,  a  teacher  with  a  restricted  idea  of  materials  and  with  restricted  materials 
is  less  likely  to  see  the  changes  she  or  he  looks  for  in  the  learner,  whether  those  changes  are 
task  specific  (as  we  say)  or  are  in  more  general  terms  of  learner  attitudes  and  strategies  toward 
learning  and  knowing. 

An  evaluation  form  such  as  the  standard  one  reproduced  in  most  Instructional 
Technology  basic  textbooks  (ie.  Heinich,  Molenda,  and  Russell,  1993)  asks  teachers  to 
examine  materials  using  a  very  narrow  range  of  issues  about  learning  and  appropriateness 
The  materials  are  evaluated  for  their  use  with  a  particular  audience,  and  it  is  assumed  that  the 
only  learning  which  needs  to  be  considered  is  that  of  the  intended  objectives  of  the  material 
Such  a  typical  form  allows  no  room  for  cultural  analysis;  it  does  not  ask  teachers  to  consider 
the  cultural  ramifications  of  the  materials,  nor  does  it  imply  in  any  way  that  those  are  issues 
a  teacher  may  want  to  consider  when  selecting  and  using  any  material  for  their  classroom. 
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A  more  involved  and  extended  approach  to  evaluation  is  one  suggested  by  Rothe 
(1991)  Questions  regarding  language,  culture,  power,  and  inclusivenesa  are  included  in  su  ch 
an  approach.  While  teachers  may  consider  this  method  more  time-consuming,  it  is  actually 
an  approach  which  helps  them  plan  the  use  of  their  materials  while  previewing  them.  No 
instructional  materials  should  be  included  in  classroom  use  without  some  discussion  of  foeir 
creation  and  intent.  Teachers  need  to  be  encouraged  by  this  evaluation  process  to  include 
discussions  about  the  use  of  language,  images,  and  the  represenUtion  of  culture  which  any 
materials  represent.  Newer  technologies,  especially,  can  "other"  people,  continuing  to 
severely  limit  the  meanings  of  what  is  learned,  and  avoid  questions  of  ecological  damage  done 
by  the  creation  and  use  of  technology. 

Just  recommending  that  teachers  reconsider  their  evaluation  of  materials  is  not 
enough,  but  it  is  one  simple  method  by  which  teachers  can  be  encouraged  to  gam  control  over 
materials  selection,  production,  utilization,  and  evaluation  for  themselves  ar.d  for  their 
students.  All  participants  in  the  education  process  need  to  be  encouraged  to  consider  the 
visual,  cultural  and  "media  literacy  issues  often  avoided  by  more  traditional  evaluation 
procedures.  The  ideas  presented  here  could  be  a  basis  upon  which  to  build  a  more  informed 
and  involved  group  activity,  seeking  support  for  inclusion  of  the  political,  economic  and 
cultural  issues  within  the  educational  milieu. 

Copies  of  handouts  are  included  after  the  references  so  that  readers  may  participate 
further  in  the  ideas  suggested  by  this  research.  This  research  project,  as  it  continues,  hopes 
to  establish  new  criteria  for  the  examination  and  use  of  visual  materials.  Teacher  educators 
and  teachers  will  promote  a  more  visually  literate  audience  with  these  activities.  Evaluation 
of  materials  becomes  one  method  by  which  to  encourage  a  reexamination  of  teaching  and 
learning  from  a  more  inclusive  cultural  dimension. 

References 

Bordwell,  D.  and  Thompson,  K.  (1986).    Film  art:    An  introduction  (2nd  ed.)  New 
York:Knopf. 

Bowers,  C.A.  (1988).   The  Cultural  dimensions  of  educational  computing.   New  York: 
Teachers  College  Press. 

Cherryholmes,  C.H.  (1988).  Power  and  criticism:  Poststructural  investigations  in  education 
(Ed).  New  York:  Teachers  College  Press. 

Devereux,  F.L.  (1935).  The  Educational  talking  picture.  Chicago,  IL:  The  University  of 
Chicago  Press,  2nd.  ed.,  pp.  204-210. 

Eisner,  E.  and  Peshkin,  A.  (1990).  Inquiry  in  Education  and  the  Continuing  Debate.  New 
York:  Teachers  College  Press. 

Ellsworth,  E.  and  Whatley,  M.H.  (1990).  (eds.)  The  Ideology  of  images  in  educational 
media,  hidden  curriculums  in  the  classroom.  New  York:  Teachers  College  Press. 

Heinich,  R.,  Molenda,  M.,  and  Russell,  J.  (1989).  Instructional  Media  and  the  New 
Technologies  of  Instruction,  3rd  edition.  New  York:  Macmillan  Publishing 
Company. 


747  V:jx 


Keir.  S.T.  (1985)  Introduction:  Asking  new  question*  .bout  technology  and  the  social 
world  of  educauon.  Educational  Communication  and  Technology.  A  Journal  of 
Theory,  Research,  and  Development,  3j,  1,  pp.3-8. 

ROtbe'  '£<2!£  Crftit r*IU*UOn  °f  edUC*ti0Ml  SOftw>re  from  »  "xi"  perspective: 
uncovering  some  hidden  assumpt.ons.  In  Hlynka,  D.  and  Belland  J.  (eds). 
Paradigms  Regained,  New  Jersey:  Educational  Technology  Press,  pp.  367-384. 

Yeaman,  A  R  L  (1990).  Sociocultund  aspects  of  computers  in  eduction.  Proceedings  of 
Selected  Research  Paper  Presentations  at  the  1991  Annual  Convention  of  the 
Association  for  Educational  Communications  and  Technology.  Ames,  Iowa-  Iowa 
State  University  and  A.E.C.T.,  pp.989-994. 

*  Portions  of  this  paper  were  previously  published  in  Braden,  R  A  Clark-Sac  l  *nA 
the  24th  Annual  Conference  of  International  Visual  Literacy. 


748  , 


Instructional  Materials  Evaluation 


Title: 


Source,  Date,  Length,  Cost: 
Subject  Area/ Audience:  


Brief  Description/Subject  Matter: 


Language  Usage 

::  feoC  :}| 

1 .  Is  the  language  used  context  specific? 

2.  Will  students  understand  the  concepts 
presented,  especially  those  presented  within 
metaphors  using  familiar  words  in  new  ways? 

3.  Is  one  point  of  view  presented  without  the 
acknowledgement  cf  other  possibilities? 

4.  Is  the  language  used  appropriate  to  the  content 
but  broad  enough  to  allow  other  points  of 
view? 

Comments: 

m 

m 

5.  Is  the  framework  for  content  selection 
explained? 

I 

6.  Is  the  design  of  the  information  obvious?  Are 
designers1  choices  explained? 

7.  Does  the  knowledge  presented  allow  flexibility 
for  student  and  teacher  involvement? 

8.  Is  this  material  marketed  in  an  easily  usable 
and  available  format? 

9.  Is  material  designed  to  reflect  mainstream 
society's  assumptions  concerning  relevant 
cultural  features  such  as  responsibility, 
education,  career,  family,  religion,  etc.? 

;ric 
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Perspectives  Coiit. 

.  Xtotofto 

.v.:.Not UT4 
1     iw  :: 

10.  Docs  the  material  promote  and  reflect  the 
distinctive  character  and  values  of  all 
cultures? 

11.  Is  this  material  directed  to  the  largest  number 
of  users  to  the  exclusion  of  minority  or 
smaller  groups  within  the  educational 
population? 

12.  Is  the  value  framework  for  this  material 
evident? 

13.  Are  the  students  or  teachers  given  the 

opportunity  to  question  or  explore  the  values 
framework  used? 

14.  Dc  tie  values  presented  interface  with  the 
values  held  by  users? 

Comments: 

CurHcuJar/Strategy  Issues 

15.  Is  this  material  promoting  effective  and 
efficient  learning  to  the  exclusion  of  more 
broadly-based  learning  outcomes? 

16.  Is  the  underlying  message  of  this  material 
that  technologically  driven  learning  is 
preferred  over  other  types  of  learning? 

17.  Is  the  material  appropriate  educationally 
rather  than  selected  as  a  result  of  some 
commercial  effort? 

18.  Is  using  this  material  the  soundest  solution  to 
the  educational  problem  involved? 

Comments: 

Derived  from  Rothe,  J.  P.  (!991).  Critical  evaluation  of  educational  software  from  a  social 
perspective:  Uncovering  some  hidden  assumptions.  In  Hlynka,  D.  and  Belland,  J. 
(eds).  Paradigms  Regained,  New  Jersey:  Educational  Technology  Press,  pp.  367-384. 

°  Nichols/Robinson/Wiegmann.  No  use  or  reproduction  without  permission. 
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THE  DESIGN  OF  EFFECTIVE  CASE  STUDY-RELATED 
LEARNING  STRATEGIES  TRAINING 


by  Alice  R  Nuttall 

Introduction 

The  puipose  of  this  paper  is  to:  (a)  describe  how  case  study  training 
materials  were  developed  for  an  experimental  research  study,  and  (b)  discuss 
effects  of  such  training* 

Because  the  research  study  (Nuttall,  1991)  examined  the  effects  of  task- 
related  learning  strategies  training  on  performance  and  motivation  in  a  case 
study  task,  it  was  necessary  to  develop  both  general  case  guidelines  and 
learning  strategy  guidelines. 

The  Need  for  Learning  Strategies  Training 

A  dearth  of  experimental  studies  on  achievement  and  motivational 
outcomes  of  training  in  self-regulated  learning  strategies  suggests  a  need  for 
research  in  this  area.  As  stated  by  Dansereau  (1985), 

...at  this  point  systematic  learning  strategy  research  is  still  in  its  infancy. 
Consequently,  it  is  impossible  to  derive  a  solid  set  of  principles  that  can 
be  used  to  plan  and  conduct  future  studies  (p.  218). 

One  needed  research  area  is  the  domain-general  vs.  domain-specific 
training  question.  For  example,  Mayer  (1988)  and  Miles  (1988)  recommend 
the  pursuance  of  domain-general  or  content-independent  research,  while 
Dansereau  (1985)  and  Cook  (1982,  cited  in  Mayer,  1988)  are  pursuing  domain- 
specific  or  content-dependent  learning  strategies  training. 

Researchers  (i.e.,  DuBois  &  Kiewra,  1989)  also  recommend  studying 
combinations  of  at  least  two  or  three  strategies  in  a  domain-  (subject-or 
content-)  based  situation  where  individual  outcomes  can  be  assessed.  Along 
that  line,  DuBois  (1988)  and  Mayer  (1988)  recommend  the  study  of  which 
strategies  make  a  difference  and  why.  Furthermore,  Mayer  (1988),  McKeachie 
(1988),  and  Weinstein  et  al.  (1988)  suggest  that  connections  between  student 
motivation  and  learning  strategies  skill  be  studied.  As  DuBois  (1988)  notes,  "It 
is  not  enough  to  analyze  GPA  changes"  (p.  17). 

Among  other  needed  research  are:  (a)  classroom  application  studies 
(Doyle,  1986;  Good  &  Tom,  1985;  Martin  &  Briggs,  1986,  Reigeluth,  1983) 
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and  (b)  the  study  of  affective  domain/motivation  outcomes ■  (Priggs. 1982; 
Como  &  Mandinach,  1983;  Martin  &  Briggs,  1986;  Reigeluth,  1983). 

learning  Strategies  Theory  and  Research  Design 

A  theoretical  framework  of  outcomes  and  influences  of  task-related 
learning  strategy  training  appears  in  Figure  1.  A  diagram  of  the  research 
design  used  to  assess  outcomes  of  learning  strategy  training  appears  in  Figure 
2. 

The  theoretical  foundation  for  learning  strategies  training  was  a  social- 
cognitive/expcctancy-value  model  of  coUege  learning  and  teaching 
by  McKeachie,  Pinliich,  Lin,  and  Smith  (1986).  This  model  hypothesized  that 

(a)  student  motivation  and  learning  strategies  knowledge  not  only  affect  each 
other  but  lead  to  both  engagement  in  learning  and  academic  performance;  and 

(b)  individual  student  characteristics  may  influence  task-related  motivation  and 
academic  performance. 

In  a  posttest-only,  two-group  experimental  design,  Associate  Degree 
introduction  to  management  students  in  a  comprehensive  state  university  were 
given  a  short  business  management  case  or  "incident"  along  with  background 
information  describing  management's  primary  functions  (planning,  organizing 
directing,  and  controlling)  and  processes  (decision  making  and  communication). 
During  four  75-minute  sessions  over  two  weeks,  54  subjects  were  asked  to 
complete  three  tasks:  (a)  list  symptoms  indicating  that  problem(s  existed;  (b) 
categorize  symptoms  as  appropriate  function  and/or  process;  and  (c)  state 
problem(s). 

Both  groups,  control  and  experimental  treatment,  received  training  in  how 
to  analyze  a  business  case  study:  in  addition,  the  treatment  group  received  task- 
related  training  in  learning  strategies  and  tactics. 

Assessments  were  made  for  three  performance  outcomes  (the  tasks  listed 
above)  and  three  task-related  motivational  outcomes:  (a)  perceived  task- 
related  self-efficacv  for  learning;  (b)  expectations  for  task-specific  academic 
success,  and  (c)  de'sire  to  continue  learning  more  about  management  in  general 
as  well  as  case  study  analysis  in  particular.  In  addition,  relationships  were 
examined  between  22  individual  characteristics,  experimental  outcomes, 
learning  strategy  use,  and  learning  strategy  value. 

Since  motivation  is,  by  definition,  consciously  or  unconsciously  decided 
goal-directed  behavior,  task-related  goal  orientation  and  self-concept  motivation 
variables  were  emphasized.  Expectancy  for  success  indicated  the  extent  to 
which  subjects  felt  they  would  obtain  high  scores  on  business  case  problem 
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F*gure  1'  0utcco>e8  ^  influences  of  tank-related  leading  strategy 
training:  A  theoretical  framework 
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tasks.  Self-efficacy  for  learning  components  included  understanding  difficult 
written  or  oral  information  presented,  learning  basic  concepts,  mastering  skills 
being  taught,  and  confidence  in  completing  assignments  and  tests  involved  in  ; 
case  problem  project 


Instructional  Materials 


Instructions  for  the  experimental  group,  Research  Group  T,  appear  in 
Appendix  A.  Other  materials  used  included:  Guidelines  for  Case  Analysis 
(Appendix  B),  including  an  example  case  (Appendix  C);  and  Learning 
Strategies  and  Hints  for  Case  Analysis  (See  two-page  excerpt  in  Appendix  D). 

A  six  page  "Supplement  to  Learning  Strategies  and  Hints  for  Case 
Analysis,"  illustrating  examples  of  learning  strategies  applications,  was  also 
provided  all  the  experimental  subjects;  this  is  available,  by  request,  from  the 
research  investigator.  All  materials,  as  well  as  the  documentation  of  their 
development,  are  also  included  in  Nuttall  (1991). 

Instructions  and  materials  that  were  given  only  to  experimental  Group  T 
are  noted  in  all  capital  letters  in  Exhibit  A,  "Instructions,"  and  refer  to  learning 
strategics  and  hints  for  case  analysis,  as  well  as  practice  worksheets  (See 
Appendix  E  for  the  example  of  a  completed  worksheet).  Both  groups  received 
all  other  materials. 


Development  of  Case  Study  Guidelines 

Cases  and  instructions  for  case  analysis  were  derived  from  a  combination 
of  published  textbooks  (Albert,  1988;  Baack,  1988;  Dietzer  &  Shilliff  1977- 
Edge  &  Coleman,  1982;  Jauch,  Coltrin,  &  Bedeian,  1989;  Schuler  &  Dalton 
1986;  Tavemier,  1988;  Wales,  Naidi,  &  Stager,  1986)  and  the  research 
investigator's  experience  teaching  a  course  entitled  Case  Studies  in  Business 
Management  Technology. 

Development  of  Learning  Strategies  Training 

Learning  strategies  training  was  based  on  strategies  that  were  not  only 
applicable  to  the  task  but  were  also  strategies  indicated  as  correlating  with 
performance  and  such  self-concept  related  motivation  dimensions  as  self- 
efficacy  and  expectancy  for  success  (See  Eison  et  al.,  1986;  Pintrich  1986 
1987;  Pintrich  &  De  Groot,  1988;  Schutz  et  al,  1989) 
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The  task  was  considered  by  experts  to  be  one  that  required  identifying 
important  points,  classifying  information,  and  stating  problems.  Ability  to  read 
and  handle  a  sheaf  of  written  instructions  and  references  was  also  needed. 

Using  definitions  and  instruments  developed  by  Pintrich  et  al.  (1988),  as 
well  as  information  processing  or  knowledge  patterns  described  by  DuBois  and 
Kiewra  (1989),  a  pilot  study  provided  lists  of  useful  task-related  learning 
strategies  and  tactics  as  we' i  as  an  assessment  of  the  relationship  to  academic 
performance  and  motivation  of  four  categories  of  strategies:  organizing, 
elaborating,  planning,  and  regulating. 

The  assessment  of  "other"  strategies  involved  the  reported  use  and  value 
of  twelve  individual  study  techniques,  rather  than  the  use  of  composite  groups 
of  strategies  described  below,  and  included  selected  resource  management 
strategies. 

Organization  strategies.  Organization  strategies  included:  (a)  identifying 
important  points  in  readings  and  notes;  (b)  outlining  reading;  (c)  outlining 
notes;  and  (d)  making  simple  charts,  diagrams,  or  tables. 

Elaboration  strategies.  Elaboration  strategies  included:  (a)  pulling 
together  information  from  different  sources  (i.e.,  lectures,  handout  information, 
readings  and  discussion);  (b)  making  connections  between  readings  and 
concepts  from  lecture;  (c)  relating  case  to  what  student  know;  (d)  writing  brief 
summaries  of  main  idea  of  case;  (e)  applying  ideas  to  cases  from  other  class 
activities  (i.e.,  lecture  and  discussion);  and  (f)  relating  ideas  from  case  studies 
to  other  courses. 

Planning  strategies.  Planning  strategies  included:  (a)  before  studying 
thoroughly,  skimming  reading  material  to  see  how  it's  organized;  (b)  making 
up  questions  about  case;  (c)  setting  goals  for  each  period  of  study  activities; 
and  (d)  thinking  through  topic  of  case  problem  rather  than  just  reading  it 

Regulating  strategies.  Regulating  strategies  included:  (a)  rereading  when 
confused;  (b)  when  difficult  readings  encountered,  changing  the  way  material  is 
read;  (c)  adapting  learning/studying  style  to  instructor's  requirements  and  style; 
and  (d)  when  confused  about  notes  or  instructions,  sorting  out  ideas  or  doing 
something  about  it 

Other  strategies.  Other  strategies  included:  (a)  having  all  materials 
organized  before  beginning  task  (organizing);  (b)  timing  work  (resource 
management);  (c)  asking  others  for  help  (resource  management);  (d)  reading 
the  case  more  than  once  (regulating);  (e)  underlining  or  highlighting  important 
points  (organizing);  (f)  writing  notes  in  the  margin  of  the  case  (organizing);  (g) 
taking  written  notes  about  the  case  (organizing);  (h)  making  a  diagram  of  the 
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cs*e  facts  (organizing);  (i)  using  a  worksheet  to  organize  symptoms 
(organizing);  (j)  writing  a  rough  draft  of  problem  statement  (organizing);  (k) 
concentrating  in  a  quiet  place  (resource  management);  and  (1)  referring 
frequently  to  instructions  and  other  materials. 

Application  of  strategies  in  instructional  material?   Most  of  the  above- 
listed  strategies  were  at  least  mentioned  in  the  instructional  materials,  although 
extensive  training  was  not  provided.  Strategies  not  mentioned  or  not  readily 
evident  in  the  materials  included:  (a)  outlining  reading  and  notes;  (b)  applying 
ideas  to  cases  from  other  class  activities;  (c)  setting  goals  for  each  period  of 
study  activities;  (d)  adapting  learning/studying  style  to  instructor's  requirements 
and  style;  and  (e)  concentrating  in  a  quiet  place. 

Discussion  of  Results 

,  ,  .7*?  trcatment  resulted  in  significant  performance  increases  in  two  tasks- 
(a)  listing  symptoms  c*  case  problem(s),  and  (b)  labelling  symptoms  according 
to  management  function  or  process.  These  outcomes  were  viewed  as  possible 
evidence  that  the  learning  strategies  treatment  had  had  at  least  some  effect  on 
outcomes. 

There  was  no  significant  increase  for  a  third  task,  writing  a  problem 
statement  However,  both  groups  performed  better  on  the  problem  task  than 
the  other  two  tasks.  Possible  reasons  for  these  results  included:  (a)  Since  there 
were  no  significant  correlations  between  problem  and  either  symptoms  or 
functions  tasks,  perhaps  different  strategies  were  needed  to  write  a  problem 
statement;  (b)  the  case  study  guidelines  may  have  served  as  a  "megastrategy" 
that  gave  both  groups  equal  training  in  solving  a  problem  task. 

In  assessing  learning  strategy  use,  two  tactics  were  used  by  the  treatment 
group  sign.ficantly  more  than  the  control  group:  (a)  referring  frequently  to 
instructions  and  other  material,  and  (b)  reading  the  case  more  than  once. 

In  addition,  post  hoc  analyses  indicated  differences  between  internal 
workings  of  the  groups  in  unique  correlations  between  learning  strategies  use 
learning  strategies  value,  self-concept  related  motivational  variables  (i  e  task- 
related  self-efficacy  and  expectancy  for  success),  performance,  and  individual 
charactensucs. 

Specifically,  results  confirmed  the  dynamism  of  continuing  intrinsic 
motivation,  which  has  been  defined  by  McCombs  (1984)  as: 

a  dynamic,  internally  mediated  set  of  metacognitive,  cognitive  and 
affective  processes  (including  expectations,  attitudes,  and  beliefs  about  the 
selt  and  the  learning  environment)  thn  can  influence  a  student's  tendency 
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to  approach,  engage  in,  expend  effort  in,  and  persist  in  learning  tasks  on  a 
continuing,  self-directed  basis  (p.  200). 

The  study  also  confirmed  that  self-efficacy  is  a  major  recurring 
motivational  factor  (See  Bandura,  1986;  Covington  &  Omehch,  1987; 
McCombs,  1986;  Schunk,  1986. 

In  addition,  results  suggested  that  learning  strategies  training  can  affect 
associations  between  variables  related  to  performance  and  motivation,  thus 
indirectly  affecting  performance  (See  Eison  et  al.,  1986;  McCombs  1984 
1987-  Pintrich,  1986,  1987;  Pintrich  &  Dc  Groot,  1988;  Schutz  et  al.,  1989), 
especially  in  the  area  of  individual  characteristics  such  as  personal  beliefs  about 
the  importance  and  value  of  both  task  and  learning  strategies. 

Conclusions 

Two  conclusions  are  evident  First,  task-related  learning  strategy  training 
is  a  viable  method  for  increasing  academic  performance. 

Secondly,  instead  of  directly  affecting  self-concept  related  motivation 
(especially  in  the  areas  of  self-efficacy  and  expectancy  for  success),  task-related 
learning  strategy  training  may  affect  synergistic  interactions  between  learning 
strategy  use  and  value,  performance,  motivation,  and  individual  characteristics. 

Implication  s  for  Practice 

Implications  for  practice  include  integrating  into  all  instructional  plans  the 
following  knowledge  of  subject  content,  learning  strategies,  and  attributions 
awareness  (knowledge  of  how  individual  characteristics  and  motivation  may 
affect  learning). 

Regarding  motivational  variables  studied,  it  is  recommended  that  self- 
efficacy  and  expectancy  for  success  continue  to  be  studied  within  the  self- 
concept  context.  Desire  to  learn  is  a  more  complex  variable  in  that  it 
incorporates  more  than  the  self -concept  context 

In  addition,  in  conducting  research  in  this  area,  two  limitations  of  the 
current  study  should  be  addressed:  use  of  self-report  instruments  and  time 
period. 

Regarding  instrumentation,  subjects  need  to  be  tested  as  to  their  actual 
knowledge  and  application  of  learning  strategies. 

Regarding  time,  adequate  time-i.e.,  a  minimum  of  three  to  six  months- 
must  be  allowed  for  appropriate  practice,  mastery,  and  motivation  for  actual 
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use  of  learning  strategies.  In  pilot  studies,  which  took  place  over  longer 
periods  of  time  (up  to  two  and  one-half  months),  subjects  enrolled  in  a  case 
studies  course  showed  significant  gains  in  planning,  self-efficacy  and 
expectancy  for  success.  This  type  of  long-term,  classroom  applications 
research  must  be  expanded. 
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INSTRUCTIONS  FOR  CASE  TRAINING  SESSIONS  -  RESEARCH  GROUP  T 

Your  task  is  to  analyze  business  management  cases.  Starting  today,  you  will 
work  on  one  case  during  each  of  the  next  three  class  periods.  During  the 
fourth  class  period  you  will  complete  a  "test  case"  and  a  questionnaire.  You 
will  not  be  asked  to  recommend  solutions.  Instead,  you  will  concentrate  on 
identifying  the  problem(s)  that  exist  in  the  case.  To  do  this,  you  will: 

(1)  List  symptoms  of  the  problem(s),  which  are  facts  that  indicate  something 
is  wrong. 

(2)  Label  each  symptom  as  one  of  the  six  management  functions  and 
processes  described  in  the  attached  "Management  Functions  and  Processes 
Review  Sheet"  (Planning,  Organizing,  Directing,  Controlling,  Communication, 
and/or  Decision  Making). 

(3)  Write  a  problem  statement  in  sentence/paragraph  format.  This  is  a 
statement  of  what  is  wrong  in  the  case  and  explains,  in  approximately  one  to 
five  sentences  (a  paragraph),  the  main  cause(s)  of  the  symptoms  of  discontent 
previously  listed.  There  may  be  more  than  one  problem  and,  because  you  may 
have  incomplete  information,  you  may  have  to  make  some  assumptions  (and 
guesses!)  in  deciding  what  are  symptoms  and  what  are  problems. 

(4)  BEFORE  YOU  WRITE  YOUR  PROBLEM  STATEMENT,  COMPLETE 
THE  WORKSHEET  ENTITLED  "ORGANIZATION  OF  SYMPTOMS  BY 
FUNCTION/PROCESS"  (THREE  BLANK  COPIES  ARE  ATTACHED  FOR 
YOUR  USE). 

More  detailed  instructions,  including  information  about  how  to  distinguish 
between  symptoms  and  problems,  appear  in  the  attached  documents  entitled 
"Guidelines  for  Case  Problem  Analysis"  and  "LEARNING  STRATEGIES  AND 
HINTS  FOR  CASE  ANALYSIS."  NOTE,  IN  PARTICULAR,  LEARNING 
STRATEGY  EXAMPLE  5  FOR  GUIDANCE  IN  COMPLETING  STEP  4 
ABOVE. 

Completing  Your  Work.  You  may  start  working  on  your  case  as  soon  as  you 
have  read  all  the  instructions  and  guidelines.  After  you  have  completed  the 
answer  sheet  attached  to  each  case,  as  well  as  the  worksheet,  you  may  obtain 
copies  of  both  Answer  Keys  from  your  instructor.  Compare  your  responses 
with  the  responses  on  the  Answer  Keys  to  determine  how  you  can  do  better  on 
the  next  case. 

(Continued  on  next  page) 
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Other  Procedures.  While  you  are  involved  in  this  project,  please  do  not  talk  t0 
anyone  about  u>  rms-rtOK.  during  or  after  chM-  ^T^gy 
nofts  with  anyone.  This  is  to  insure  that  everyone  has  an  equal  chance  at 
doing  well  on  the  cases.  Keep  these  instructions,  including  the  Management 
Functions  and  Processes  Review  Sheet,  Guidelines,  WORKSHEETS 
LEARNING  STRATEGIES  AND  HINTS,  and  all  answer  keys,  for  reference  in 
doing  your  other  case  analyses. 


Materials  to  Qhtain  from  Instructor:  Case  A,  Acme  Machine  Company  (for 
first  day);  Case  B,  Management  by  Rushing  Around  (for  second  day); 
Case  C,  Conflict  on  the  Plant  Floor  (for  third  day);  and  blank  answer  sheets 
and  answer  keys  for  each  case  (one  case  per  class  period) 
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GUIDELINES  FOR  CASE  PROBLEM  ANALYSIS 


Introduction 

The  cases  you  will  be  studying  are  examples  .of  situations  experienced  in  the 
business  world.  Something  has  happened  that  has  caused  one  or  more 
problems,  and  management  must  decide  what  to  do  to  improve  the  situation. 
In  order  to  recommend  solutions  to  any  problem,  however,  management  must 
determine  what  the  problem  is;  for  without  knowing  what  problem  needs  to  be 
solved,  any  recommended  solutions  would  be  merely  guesses. 

As  Gerald  Tavernier,  a  management  consultant  has  noted,  more  and  more 
companies  are  realizing  that  the  cost  of  trying  to  solve  the  wrong  problems  can 
be  very  high,  and  many  companies  have  started  formal  training  programs  for 
managers  on  problem  solving,  "with  particular  emphasis  on  problem 
identification." 

One  example  of  poor  problem  identification  is  the  situation  where  a  company 
thought  its  problem  was  high  inventory,  when  actually  the  inventory  was 
uneven;  the  stock  levels  for  factory  supplies  of  each  item  were  wrong,  causing 
production  to  get  behind  schedule,  even  though  there  were  surpluses  of  some 
materials.  Another  example  was  a  West  German  firm  that  blamed  its  drop  in 
exports  on  the  high  value  of  its  country's  currency,  when  the  real  cause  was 
faulty  engineering  and  a  failure  to  meet  market  demand. 

Guidelines  about  how  to  complete  your  cases  problem  instructions  appear 
below.  The  attached  case  "No  Business  Like  Snow  Business"  will  be  used  as 
an  overall  case  example.  Refer  to  the  answer  key  attached  to  these  guidelines 
for  examples  of  correctly  stated  symptoms,  functions,  and  problem. 


1.  Listing  Symptoms 

Symptoms  are  evidence  that  a  problem  exists.  List  all  facts  that  indicate  that 
something  is  wrong  or  where  a  situation  exists  that  could  be  improved.  Be 
very  specific  in  noting  unusual  behavior,  conflicts,  or  changes.  For  example, 
instead  of  stating  "poor  morale,"  state  that  "employees  were  refusing  to  do  their 
work  or  breaking  the  machinery."  You  will  determine  later  whether  or  not  this 
behavior  was  caused  by  poor  morale  or  something  else. 

In  addition,  don't  make  value  judgments  yet;  just  describe  what  happened. 
Don't  just  say,  "He  didn't  communicate  well."  Instead,  list  specific  behavior 
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Guidelines  for  Case  Problem  Analysis  (Page  2) 

you  observed  that  led  you  to  this  conclusion«i.e.,  "there  was  not  much  eye 
contact,  the  person's  voice  sounded  harsh,  the  person  fidgeted  in  his  seat, 
folded  his  arms,  looked  away  and  sighed,  interrupted  others  when  they  were 
talking,  etc." 

Also  separate  important  facts  from  unessential  facts.  In  the  "Snow  Business" 
case,  information  in  the  second  paragraph  about  the  SBA  consultant  was 
background  information  to  set  a  general  scenario  and  was  not  essential  to  the 
identification  of  case  problems. 

Finally,  be  alert  to  case  terminology  where  symptoms  may  be  called  problems. 
For  instance,  in  the  "Snow  Business"  case,  the  last  paragraph  states,  "The 
Turn's  questions  and  problems  centered  around  the  only  cash  register..."  Here, 
the  cash  register  was  a  symptom  of  a  problem,  not  the  problem  itself.  The  root 
problems  were  planning  and  directing,  as  related  to  the  cash  register. 
Controlling  was  not  a  root  problem  because  it  was  caused  by  something  else- 

1.  e.,  poor  policies  and  procedures  (planning  functions). 

2.  Label  Symptoms  According  to  Management  Function  or  Piters 

Now  that  you  have  a  list  of  symptoms  indicating  that  one  or  more  problems 
exist,  label  each  of  these  symptoms  as  one  of  the  following  six  functions  or 
processes  of  management:  Planning,  Organizing,  Directing,  Controlling, 
Communication,  and  Decision  Making.  These  are  described  in  your  review 
sheet  This  process  will  help  pinpoint  the  main  problem  areas  in  the  case. 
Note:  It  is  possible  that  one  fact  could  represent  more  than  one  management 
function  or  process.  Before  assigning  more  than  one  function,  however,  break 
down  facts  into  smaller  units  if  possible. 

3.  Write  Problem  Statement 

The  problem  statement  explains  the  main  cause(s)  of  symptoms  of  discontent 
previously  listed.  Approximately  one  to  five  sentences  (a  paragraph)  should  be 
adequate  to  tell  management  what  is  wrong  in  a  situation  and  what  needs  to  be 
corrected.  Concentrate  on  the  major  functions  and  processes  of  management 
you  used  to  classify/label  your  symptoms. 

As  advice  in  writing  your  problem  statement,  note  that:  (a)  Since  symptoms 
may  have  many  causes,  there  may  be  more  than  more  problem;  and  (b)  the 
problems  you  list  should  be  significant  ones.  In  the  "Snow  Business"  case,  for 
example,  the  shortages 
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Guidelines  for  Case  Problem  Analysis  (Page  3) 

in  the  cash  register  could  have  been  due  to  at  least  two  different  problems: 
(1)  Lack  of  appropriate  policy  and  procedures  (a  planning  problem)  and  (2) 
Lack  of  appropriate  controls  over  the  disbursement  of  monies  (a  controlling 
problem).  In  this  case,  lack  of  appropriate  controls  was  caused  by  lack  of 
appropriate  policy  and  procedures,  which  makes  planning  the  significant 
problem  area. 

In  determining  whether  a  problem  is  significant,  ask  yourself  how  many 
symptoms  can  be  explained  by  it  The  more  symptoms  a  problem  can  explain, 
the  more  significant  it  is.  In  case  study  analysis,  the  most  significant  problems 
arc  called  major  problems,  while  less  significant  problems  are  called  minor 
problems.  For  the  purposes  of  your  task,  you  will  concentrate  on  major 
problems. 

Distinguishing  between  Symptoms  and  Problems 

A  problem  exists  when  there  is  a  gap  between  an  existing  condition  and  a 
desired  situation.  Symptoms  are  the  facts  showing  that  something  is  wrong- 
that  a  problem  exists.  In  identifying  a  problem,  you  might  pretend  that  you  are 
a  detective  looking  for  clues  that  something  is  wrong,  in  which  case  these  clues 
would  be  symptoms  of  a  problem. 

Once  you  have  identified  the  symptoms,  you  then  make  guesses  on  what 
caused  them,  since  the  cause  of  the  symptoms  is  the  problem.  For  example,  a 
patient  tells  a  doctor  he  has  a  high  temperature  and  headache  (a  symptom),  but 
the  problem  that  is  causing  the  headache  could  vary  from  a  mild  case  of  flu  to 
a  severe  case  of  brain  infection. 

Examples  of  symptoms  of  business  problems  that  you  might  be  familiar  with 
include  high  absenteeism,  tardiness,  and  complaints  about  the  work 
environment.  Depending  on  other  facts  in  a  particular  case  and  assumptions 
you  make,  the  problem  could  be  related  to  poor  communication,  bad  leadership, 
inadequate  policy  or  planning,  inefficient  organization  of  the  work  station,  low 
pay,  unfair  standards  of  performance,  etc. 


3/19/90 

Alice  E.  Nuttall 
Kent  State  University 
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MANAGEMENT  FUNCTIONS  AND  PROCESSES  REVIEW  SHEET 


Management  Functions 

PLANNING  -  Determines  what  needs  to  be  accomplished  and  determines  how 
and  when  it  will  be  done;  anticipates  needs  in  establishing  goals,  objectives, 
policy,  procedures,  budgets,  and  schedules. 

ORGANIZING  -  Arranges  and  coordinates  resources  so  that  objectives  can  be 
met;  effectively  organizes  staff,  materials  and  equipment,  methods,  procedures, 
and  financial  resources  to  meet  organization's  objectives. 
Sets  up  organizational  structure  and  chain  of  command. 

DIRECTING  -  Through  leadership  and  delegation  skills,  provides  guidance  to 
subordinates  so  they  can  accomplish  their  assigned  work.  Also  recognizes 
individual  differences  in  people  and  points  of  view  and  creates  a  motivational 
atmosphere  that  will  build  employee  morale. 

• 

In  the  area  of  leadership,  elicits  enthusiastic  cooperation  and  creative  initiative 
from  both  subordinates  and  associates.  Applies  appropriate  style  of  leadership 
to  given  situation;  knows  when  to  be  directive  and  when  to  be  a  participative 
manager,  whether  in  a  small  or  large  business  organization. 

In  the  area  of  delegation,  delegates  responsibilities  as  appropriate  and 
effectively  utilizes  subordinates'  skills.  Knows  how  to  clearly  define  and 
follow  up  on  delegated  responsibilities  to  the  degree  that  subordinates  carry  out 
assignments  even  when  not  under  constant  supervision. 

In  carrying  out  management  functions,  knows  how  to  make  quantitative 
applications  and  consider  both  behavioral  dimensions  and  environmental 
conditions  of  a  given  situation.  Is  responsible  for  organization's  financial 
situation. 


CONTROLLING  -  Appropriately  measures  results  against  established 
objectives  and  standards  (i.e.,  quantity,  quality,  timeliness,  and  cost);  monitors 
desired  performance  against  objectives  for  both  self  and  subordinates,  and 
corrects  procedures  and  performance  as  necessary. 

Processes  Used  to  Perform  Management  Functions 

COMMUNICATION  -  Demonstrates  effective  writing,  speaking,  and  listening 
skills  in  communicating  with  all  levels  of  employees.  Is  sensitive  to  the  role 

(Continued  on  next  page) 
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Management  Functions  and  Processes  Review  Sheet,  Page  2 

personal  values  and  attitudes,  and  other  personal  characteristics  play  in 
interpersonal  interactions* 

DECISION  MAKING  -  In  solving  problems  and  making  decisions,  shows 
discretion  and  arrives  at  sound  common  sense  decisions.  Uses  both  fact  and 
intuition-based  problem  solving  strategies;  demonstrates  logical,  clear,  and 
consistent  thought,  as  well  as  ability  to  generalize  what  is  learned  from  one 
situation  to  another. 


2/18/90 

Alice  Nuttall 

Kent  State  University 


APPENDIX  C 

EXAMPLE  CASE:  NO  BUSINESS  LIKE  SNOW  BUSINESS 

Bill  and  Janet  Schneeman  are  co-owners  of  the  Olympic  Ski  Haus,  a  new  and 
small  sporting  goods  store  that  specializes  in  ski  and  tennis  equipment  along 
with  apparel  accessories  for  both  these  sports.  During  the  stages  of 
incorporation,  the  Schneemans  applied  for  a  Small  Business  Administration 
loan  of  $50,000.  As  a  condition  of  receiving  the  loan,  Olympic  Ski  Haus 
agreed  to  the  services  of  an  SBA  consultant-^  retired  business  executive  with 
retailing  experience. 

During  his  investigative  interview,  the  SBA  consultant  discovered,  among  other 
things,  that  when  Bill  and  Janet  incorporated  the  business,  both  agreed  there 
would  be  no  boss.  "We  share  authority  and  responsibility  equally  here,"  they 
told  the  consultant. 

Furthermore,  they  advised  him  that  in  the  interests  of  equality,  it  was  agreed 
that  each  party  could  independently  make  decisions  and  set  policies  for  the 
management  of  the  store.  Each  could  give  orders  and  instructions  to  the  store's 

(Continued  on  next  page) 
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Appendix  C,  Example  Case:  No  Business  Like  Snow  Business,  Page  2 

two  employees  with  the  mutual  understanding  that,  if  any  employee  had 
questions  or  problems,  he  or  she  could  appeal  to  either  owner  for  help  or 
advice. 

"We  are  too  small  an  outfit  to  be  concerned  about  such  things  as  objectives  and 
policies  here  at  Olympic.  'No  business  like  snow  business9  is  our  store's 
objective.  We  take  each  day  as  it  comes  and  try  to  increase  our  business 
volume. 

"As  for  policies,  we  handle  each  situation  as  it  develops-because  each  is 
different~and  whoever  is  here  will  decide  then  and  there  what  to  do.  If  the 
hired  help  would  get  different  answers  or  advice,  it  wouldn't  really  matter  since 
we're  so  small.  Right  Janet? 

"As  for  any  market  research  to  determine  what  our  store  potential  is  in  this 
trading  area,  that's  for  big  firms.  We  have  neither  the  time  nor  the  help  to  do 
this."' 

The  SBA  consultant  discovered  that  many  of  the  firm's  questions  and  problems 
centered  around  the  only  cash  register  at  Olympic.  Both  Bill  and  Janet  take 
cash  from  the  register  to  cover  incidental  expenses,  including  meals  and 
beverages  consumed  during  store  hours.  At  the  close  of  day  either  one  present 
will  count  and  balance  the  cash  receipts  against  the  register  totals.  Inevitably 
the  cash  register  is  short,  and  each  day  the  sales  clerks  are  notified  of  the 
shortage  amount.  Consequently,  the  consultant  observed  that  the  clerks  feel 
mistrusted  and  hesitate  to  operate  the  register  for  fear  of  being  accused  of 
either  theft  or  an  inaccurate  sales  transaction.  As  a  result,  both  employee  sales 
and  motivation  seem  low. 

Source:  Bernard  A.  Deitzer  and  Karl  A.  Shilliff.  Contemporary  Management 
Incidents.  Columbus,  OH:  Grid,  Inc.  (1977),  pp.  20-21.  Copyright  by 
Bernard  A,  Deitzer.  Used  by  permission. 
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ANSWER  KEY 
EXAMPLE  CASE:  NO  BUSINESS  LIKE  SNOW  BUSINESS 
(One  point  for  each  of  12  symptoms  and  12  functions) 


A,  Symptoms 

1.  No  one  boss;  shared  authority 
(Don't  know  who  to  listen 
to;  no  leadership) 

2.  Shared  responsibility  (No  boss; 
each  gives  orders) 

3.  Independent  or  inadequate  policy 
making 

4.  Independent  decision  making 
(Situational  individual 
decision  making) 

5.  No  objectives/goals 
(Day-to-day  outlook) 

6.  No  policies/procedures/ 
rules  (Day-to-day  outlook) 

7.  No  makert  planning/research 
(Research  alone  =  1/2  pt) 

8.  Shortages  in  cash  register 

9.  Clerks  feel  mistrusted 
(Won't  operate  register) 

10.  Low  employee  sales  (Low 
morale  and  confidence) 
confidence) 

11.  Low  employee  motivation 


12.  Both  owners  take  cash 
from  drawer  and  balance 
(inaccuarate  accounting  cash) 


B.  Management  Function/Process 
Organizing 

(Also  accept:  Organizing 
and  Directing) 


Directing 
Planning 

Decision  Making 

Planning 

Planning 

Paining 
Controlling 

Directing  (Also  accept: 
Directing  and  Communicating) 

Directing 
(Also  accept: 

Directing  and  Communicating) 

Directing  (Also  accept: 
Directing  and  Communicating) 

Controlling 
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Example  Answer  Key 

No  Business  Like  Snow  Business 

Page  2 


C.  Problem  Statement  (6  points) 

There  is  a  need  for  better  directing  and  planning.  Specifically, 
management  lacks:  (1)  clear,  unified  authority  (a  directing  function);  (2) 
clear  plans  and  objectives  (a  planning  function);  and  (3)  policies  or  standards 
for  operation  that  all  employees  can  adhere  to  on  an  equal  basis  (both 
planning  and  directing  functions).  These  policies  and  standards  will  provide 
controls  needed. 

3  KEY  POINTS  (2  points  each): 

1.  DIRECTING  (LEADERSHIP  AND  COMMUNICATION  WITH 
EMPLOYEES) 

2.  CLEAR,  UNIFIED  AUTHORITY  (FORMAL  MANAGEMENT 
STRUCTURE/ORGANIZATION) 

3.  PL ANNNING/PLANS :  OBJECTIVES  OR  GOALS;  POLICIES 
STANDARDS,  AND  CONTROLS 


NOTES: 

-Give  only  one  point  for  two  symptoms  listed  together. 
--Give  partial  credit  when  more  than  one  function  is  listed. 
--No  points  are  subtracted  for  wrong  answers. 

3/19/90 

Key  developed  12/31/89  via  input  from  R.  Davis,  R.  Nay,  R.  Peterson,  H. 
Tritt,  &  E.  Villella  (60%  agreement  required). 

Final  revisions  and  decisions  about  unclear  items  on  key  made  in  unanimous 
agreement  by:  J.  Gailey,  H.  Tritt,  and  E.  Villella  at  1/2/90  meeting  chaired 
by  A.  Nuttall. 
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LEARNING  STRATEGIES  AND  HINTS  FOR  CASE  ANALYSIS 

The  learning  strategies  described  herein  will  help  you  obtain  higher  grades 
on  each  case  task.  Examples  of  these  strategies  are  attached. 

1.  Plan  Your  Work 

a.  Before  you  begin,  be  sure  you  have  all  your  materials  and  notes  together 
and  that  you  understand  the  assignment.  Assemble  all  the  guidelines  and 
other  handout  material  provided  by  the  instructor,  and  read  all  instructions, 
including  this  handout  on  Learning  Strategies. 

b.  Look  at  the  clock  to  determine  how  much  time  you  have,  but  don't  hurry 
through  the  assignment 

c.  If  you  don't  understand  any  part  of  the  assignment,  get  help.  Use  a 
dictionary  and/or  ask  your  teacher. 

2.  Read  Case  for  Understanding 

a.  Skim  the  case  by  reading  quickly  without  taking  notes;  this  will  give  you 
a  brief  overview  of  the  case. 

b.  Go  back  over  the  case  a  second  or  third  time,  reading  carefully  and 
highlighting  important  points  (underline,  marie  with  a  light  colored  thick  felt 
pen,  and/or  write  notes  in  the  margin).  SEE  EXAMPLE  1. 

While  you're  reading,  concentrate  on  the  case  facts  and  keep  an  open  mind. 
Don't  make  value  judgments--i.e.,  good,  bad,  true,  untrue,  etc.-about 
evidence,  people,  organizations,  or  actions.  For  example,  if  you  are  "pro- 
union"  and  management  is  not  accepting  union  demands  in  the  case,  do  not 
let  personal  feelings  affect  your  thinking.  Make  sure  you  consider  all 
possible  viewpoints.  Later  on,  you  will  make  judgments  about  the  value  or 
relevance  to  the  case  of  each  of  these  facts. 

c.  Take  written  notes  about  the  facts  of  the  case.  Ask  yourself  who,  what, 
when,  where,  why,  and  how  questions:  Who  is  involved?  What  is 
involved?  What  happened?  When  did  it  happen?  Where  did  it  happen? 
Why  did  it  happen?  How  frequent,  serious,  and  extensive  is  it?  List  key 
names,  dates,  events,  or  exhibits  for  later  reference. 

SEE  EXAMPLE  2. 
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Learning  Strategies  and  Hints,  Page  2 

Think  of  the  case  as  a  play  with  actors  (the  "who"),  props  (the  "what  is 
involved"),  action  (the  "what  happened"),  scenes  ("when  and  where  it 
happened"),  causes  ("why  did  it  happen"),  and  significance  of  consequences 
("how  frequent,  serious  and  extensive  is  it?").  SEE  EXAMPLE  3. 

d.  If  the  reading  is  confusing,  try  going  over  it  again  more  slowly  to 
understand  it. 

3.  Locating  Symptoms 

a.  Use  your  notes  and  highlighting  as  your  main  source  for  locating 
symptoms  so  you  don't  have  to  waste  time  by  rereading  the  whole  case. 

b.  To  separate  symptoms  from  general  case  facts,  code  notes  or 
underlining/highlighting  by  some  method  such  as  circling  or  using  check 
marks.  Example:  Use  underlining  for  major  symptoms  and  parentheses  for 
general  background  information.  SEE  EXAMPLE  1. 

4.  Classifying  Symptoms  according  to  Management  Function  or  Process 

Carefully  distinguish  among  functions  and  processes  by  using  underlining  or 
highlighting  on  your  Review  Sheet  to  be  sure  that  you  understand  the 
definitions  of  functions  and  processes. 

5.  Use  a  Diagram  to  Connect  Symptoms  and  Main  Events 

A  rough  draft  flowchart  or  "cluster  diagram"  will  help  you  visualize  the 
connections  between  symptoms  and  main  events,  leading  to  possible  causes 
of  theses  symptoms.  Simply  draw  lines  and  arrows  between  events  that 
show  possible  causes  and  effects.  A  straight  line  might  indicate  a 
relationship,  although  not  necessarily  cause/effect,  while  an  arrow  could 
indicate  cause/effect.  SEE  EXAMPLE  4. 

6.  Writing  the  Problem  Statement 

a.  Draw  on  your  own  personal  work  experiences  or  community  activities, 
work  experiences  of  your  friends  and  family,  and  similar  situations  discussed 
in  your  business  classes. 

b.  Use  your  diagram  to  show  the  relationships  of  symptoms  and  other  facts 
in  the  case.  Ask  what's  leading  to  what.  What  situation/factor  is  causing 
most  of  the  other  factors?  SEE  EXAMPLE  4. 
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c.  On  your  Organization  of  Symptoms  by  Function/Process  Worksheet,  list 
all  the  symptoms  that  relate  to  each  category,  then  count  the  number  of 
symptoms  listed  for  each  function/process.  Ask  which  management 
functions  or  processes  have  the  most  symptoms  listed. 

SEE  EXAMPLE  5. 

d.  Try  brainstorming;  write  out  your  thoughts  as  they  occur,  then  rewrite 
later.  In  your  own  words,  based  on  your  own  experiences  and  your  diagram 
of  events  (See  Example  4),  try  to  explain  what  the  connections  between 
symptoms  are.  Write  a  draft  of  what  happened,  along  with  reasons.  This  is 
called  elaboration.  SEF  EXAMPLE  6. 

e.  Rewrite  your  draft  into  a  brief  problem  statement  (See  Answer  Key). 
Using  the  management  functions  categories  listed  in  your  Worksheet 
(Example  S),  state  what  the  general  problems  are.  Then  back  up  this 
statement  with  examples.  Say,  "These  problems  are  evidenced  by...." 

an/KSU  3/19/90 


ERLC 


779    v  > 


APPENDIX  E 

EXAMPLE  5 

WORKSHEET:  ORGANIZATION  OF  SYMPTOMS  BY 
FUNCTION/PROCESS 

PRETEST  CASE:  NO  BUSINESS  LIKE  SNOW  BUSINESS 

PLANNING: 

1.  No  objectives  or  goals  (day-to-day  outlook) 

2.  No  policies,  rules  or  procedures  (day-to-day  outlook) 

3.  No  market  planning  or  research 
ORGANIZING: 

1.  Unclear  chain  of  command;  two  bosses  have  authority 
DIRECTING: 

1.  No  one  boss;  no  unity  of  command 

2.  Clerks  feel  mistrusted  and  won't  operate  register 

3.  Low  employee  sales 

4.  Low  employee  motivation 
CONTROLLING: 

1.  Shortages  in  cash  register 

2.  Both  owners  take  cash  from  drawer  and  balance 
DECISION  MAKING: 

1.  Independent,  situational  decision  making  by  co-owners 
COMMUNICATING: 

1.  Employees  don't  know  which  boss  to  listen  to  or  approach 
an,  2/24/90 
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The  Effects  of  Group  Computer-Based  Instruction  and  Learning  style  on 

Achievement  and  Attitude 

the  IcmtS^^SS^?^^^  induced  and  used  in  the  classroom  in  an  effort  to  facilitate 
using  a  variety  of  delive'ry  systems X  SKflSS*'  llCS'8nCrS  Wh°  mM 

preferred  methods  which  utilized  individEu^  0thCrS 
,am« 

intervening  ™M ^vcrncm  and  attitudes  of  students  be  affected  by  other 

asked  ourselves  would  a  thudS^^  W,lh,n  a  "'"'Putcr-bascd  learning  environment.  Wc 

CBI  by  individuals?  Furthe" w "<£ woul ^^^^JS^^ ^  ^  °'  Wilh  ta*l**« ■»  «* 
■n  a  computer-based  lesson?  P  lyP°  °f  lCmKt  bcncl  11  ",orc  h*  Sr(lUP     independent  learning 

Background  of  the  Study 

Commiter-BasKd  Instruction 

as  effective  or  can  be  more  effective  in manv SJn^Sff?"^1  !nslruncUon'  "tellers  have  found  (hat  CBI  is 
Walbcrg,  1985;  Kulik  4°S5  K S RS t  KblCK  fi?  J  ?ran^98,5  Nicmi<*  & 
l<>83;  Kulik,  Bangert,  &  Williams  1983-  Kulik  Kulik  A  fVihm  ii»m  f'  i^i.  o  &  Ban8crl-Drowns,  1985a;  Kulik, 
reported  that,  in  some  cases,  retenUon  may  1^ SigWy ^^JST  '      ^  "d  Wi'UamS  0983)  a'S° 

effects  on*£^  rosul*  havc  shown  ■»  -™« 
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&  Bcn-Kohav,  1980;  Kulik  ci  al.,  1987, 1986, 1985, 1983),  as  well  toward  instruction  in  general  (Atkinson,  1984; 
Kulik.ctal.,  1*987,  1986, 1985b,  1983). 

Cooperative  Learning 

Johnson  and  Johnson  (1987)  define  cooperative  learning  as  the  instructional  use  of  small  groups  so  that  students 
work  together  to  maximize  their  own  and  each  other's  learning.  A  variety  of  methods  of  cooperative ^earning  abound  in 
The  literature  and  practice  (Johnson  &  Johnson,  1987;  Slavin,  1980a,  1980b;  DeVries  &  Edwards,  1974;  Kagan  1985). 
Each  of  these  methods  vary  by  the  number  of  students  per  group,  type  of  learning  goals  and  tasks,  and  the  specific 
training  methodology  and  procedures  employed. 

The  use  of  cooperative  learning  techniques  in  conjunction  with  CBI  has  been  seen  by  some  scholars  as  desirable 
(Dalton,  Hannafin,  &  Hooper,  1989;  Hooper  &  Hannafin,  1988a,b;  Johnson,  Johnson  &  Stannc  1985;  Johnson, 
Johnson  &  Stannc,  1986;  Webb,  1987).  It  has  been  found  that  the  use  of  cooperative  learning  and  CBI  have  had 
positive  effects  on  performance  (Johnson,  Maruyama,  Johnson,  Nelson,  &  Skon,  1981;  Johnson,  Johnson,  &  Stannc, 
1985-  Slavin  1983b;  Dalton,  Hannafin,  &  Hooper,  1989).  Although  much  research  suggests  that  computer-based 
cooperative  learning  may  promote  higher  performance  than  individual  instruction,  these  findings  are  not  conclusive. 
Carrier  and  Sales  (1987)  found  no  differences  in  the  performance  scores  of  students  working  in  groups  and  those  work.ng 
individually.  Similarly,  Sherman  (1988)  found  no  significant  difference  between  the  performance  scores  of  students 
learning  in  a  cooperative  environment  and  those  learning  in  an  individual  environment. 

In  addition  to  effects  on  performance,  the  use  of  cooperative  learning  techniques  has  had  positive  effects  on 
student  attitudes  (Mcvarcch,  Stern,  &  Lcvita,  1987;  Slavin  &  Karwcil,  1981;  Johnson,  Johnson,  &  Stannc  1986). 
Again  it  must  be  noted  that  not  all  studies  report  positive  outcomes  in  terms  of  student  attitudes  (Slavin  1990; 
Moskowit/,  Malvin,  Schaclfcr,  &  Schaps,1983;  Madden  &  Slavin,  1983;  Slavin,  Leavey,  &  Madden,  1984;  Johnson, 
Johnson,  Scott,  &  Ramolac,  1985). 

Although  research  on  the  use  of  cooperative  learning  in  CBI  environments  have  had  mixed  results,  necessity 
may  be  the  main  reason  for  its  continued  use  in  computer  environments.  Typical  computer  classroom/lab 'Settings  rarely 
provide  each  individual  student  with  a  computer  (U.S.  Congress,  Office  of  Technology  Assessment  1988).  Thus,  if 
students  arc  to  spend  a  significant  amount  of  lime  with  CBI,  they  may  have  to  do  so  as  a  member  of  a  group  (Trowbridge 
&  Durin,  1984).  Because  of  this  restriction,  effective  instructional  methods  for  helping  students  work  in  groups  needs  to 
lie  found. 

Based  on  past  research  it  is  noted,  however,  that  other  intervening  factors  such  as  learning  style  could  affect  both 
ncrformancc  and  attitudes  of  students  learning  by  cooperative  methods.  For  example,  Swing  and  Peterson  (1982),  in  their 
investigation  of  student  ability  and  student  behaviors  during  small-group  interaction,  found  that  performance  was  related 
to  student  ability.  Similarly,  Webb  (1982)  found  that  performance  was  related  to  ability,  but  in  addition  to  group 
composition,  gender,  and  personality.  Finally,  Hall,  Rocklin,  Danscreau,  Skaggs,  O'Donncll,  Lambiottc  and  Young 
(1988)  administered  nine  individual  difference  instruments,  measuring  a  number  of  characteristics.  They  also  lound  thai 
individual  differences  may  have  an  effect  on  performance  in  cooperative  learning  situations.  One  specific  individual 
difference  which  may  affect  performance  in  cooperative  group  learning  is  the  learning  style  of  introvert  and  extrovert. 

learning  Stvlc  ... 

Learning  style  differences  found  in  individuals  may  affect  both  performance  and  attitudes  in  cooperative  group 
learning  as  well  as  in  individual  learning  situations.  Learning  style  refers  to  an  individual's  characteristic  mode  of 
gaining  processing,  and  storing  information  (Davidson,  1990).  Although  some  researchers  question  its  usefulness 
(Cohen  Hyman  Ashcroft,  &  Loveless,  1989;  Scwall,  1986;  Freedman  &  Stumpf,  1980),  others  advocate  the  use  of 
learning  style  information  when  investigating  the  instructional  process  (Davidson  &  Savcnyc,  1990;  Davidson  Savcnye. 
&  Orr,  1992;  Dunn,  Bcaudry,  &  Klavas,  1989;  Grcgorc,  1979;  Keefe,  1979;  Carrier,  Newell,  &  Langc,  1982;  Fi/zcll, 
l984;  Claria'na  &  Smith,  1988). 

To  date  there  is  little  research  on  the  relationship  of  learning  style  with  student  performance  on  CBI.  A  recent 
study  by  Davidson  and  Savcnyc  (1990;  and  with  Orr,  1992)  suggests  there  is  a  relationship  between  learning  style  and 
performance  on  computer-based  instructional  tasks.  Sharma  (1987)  also  found  evidence  to  support  the  consideration  ol 
learning  style  in  the  planning  of  instructional  programs.  As  Sharma  points  out,  the  process  of  identifying  individual 
learning  styles  and  using  this  information  is  not  a  new  idea,  however,  using  it  in  terms  of  CBI  warrants  more  study, 
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Likewise,  few  research  studies  have  investigated  the  effects  of  learning  style  on  attitudes,  especially  in 
conjunction  with  a  computer-based  lesson.  However,  one  study  by  Charkins,  OToole,  and  Wetzel  (1985)  showed  a 
relationship  between  learning  style  and  attitude.  Canino  and  Cicchelli  (1988)  also  found  a  significant  difference  in 
attitudes  of  subjects  involved  based  on  their  identified  learning  style.  Finally,  research  by  Davidson  and  Savcnyc  (1990) 
also  suggests  that  there  is  a  relationship  between  learning  style  and  attitude  in  CBI. 

Currently,  research  does  not  strongly  support  the  consideration  of  learning  style  alone  as  a  variable  for 
improving  instruction.  However,  learning  style  combined  with  other  instructional  variables,  such  as  CBI  and/or 
cooperative  learning,  may  provide  insight  into  which  instructional  strategics  arc  most  effective  for  different  types  of 
earners.  It  is  possible  that  assignment  of  students  to  group  or  individual  instruction,  based  on  their  style  preference  wil 
lead  to  improved  performance  as  well  as  improved  attitudes. 

Statement  of  the  Problem 

Based  on  descriptions  provided  by  learning  style  instruments,  it  was  hypothesized  that  some  particular  Icarninu 
styles  among  individuals  would  prefer  working  in  groups  whereas  others  with  differing  styles  will  prefer  workinn 
individually.  Mc.sgcier  and  Murphy  (1987)  offer  a  learning  style  instrument  that  classifies  learners  as  cxtravcas  (Note- 
Mc.sgcicr  and  Murphy  s  spelling  of  the  term),  introverts,  or  undetermined.  Because  of  these  differences  in  learning  style 
the  focus  of  this  research  was  to  study  the  effects  of  die  learning  style  of  extroversion-introversion  in  combination  will.  ' 
cwperativc  learning  on  both  performance  and  attitude  in  a  computer-based  instructional  environment. 

The  purpose  of  this  investigation  was  to  determine  the  implications  of  inslrucliofal  delivery  (c<x>pcr«livc 
learmng)  and  learning  style  (inrovcrt-cxtravcrt)  for  the  design  of  CBI  environments.  Specifically,  the  study  examined  ihe 
elfccts  of  instructional  delivery  and  learning  style  on  performance,  as  well  as  attitudes. 

Methodology 

Subjects 

Subjects  were  190  elementary  students  ranging  from  nine  to  twelve  years  of  age  in  either  the  fourth  or  tilth 
grade  a  an  affluent  school  district  in  Austin,  Texas.  Participation  in  the  study  was  on  a  voluntary  basis,  ret  u" ring 
parental  consent  Subjects  were  randomly  assigned  to  one  of  six  experimental  conditions  stratified  by  their  individual 
learning  style.  Approximately  equal  numbers  of  fourth  and  fifth  grade  subjects  would  be  assigned  to  die  cooperative 
group  and  individual  conditions.  1  1 

Treatment  Materials 

Curative  Training,  The  cooperative  learning  methodology  chosen  for  this  study  was  Learning  Together 
Ctohnson  &  Johnson.  1975).  Learning  Together  is  a  cooperative  learning  method  emphasizing  five  major  elements- 
positive  mterdependci.ee,  face-toface  interaction,  individual  accountability,  interpersonal  and  small  group  skills  and 

mZTk1?*-  i*™"*,  T°geihCr  ch0Se"  baSed  on  its  abi!ity lo  te  adaPlcd  t°  existing curriculum 
materials.  (Note:  It  was  only  used  with  the  CGI  group.)  swum-uiwu 

CBI  LfiSSQlL  The  instructional  materials  used  in  this  study  included  a  computer-based  tutorial  on  astronomy 
specifically  on  mapping  the  night  sky.  The  title  of  the  software  war.  "An  Introduction  To  The  Night  Sky"  (Orange 
r^fto^SJ^^^  oj  astronomy  were  a  part  of  the  existing  school  curriculum  ul  the  fourth  and 
li  i  grade  level,  therefore,  the  csson  provided  a  logical  extension  to  the  district's  required  objectives.  The  software  ran  on 
Ape  lie  and  Apple  llgs  machines  which  were  housed  in  the  school  computer  lab;  there  were  eighteen  compmc  n 
«k  1.1  on  to  the  software,  the  researcher  developed  a  handout  of  terms  and  procedures  that  were  included  in  the  proSa  to 
assist  students  in  pulling  out  relevant  information.  1  y 

Performance  Measures.  Performance  was  measured  by  a  29-ilcm  multiple  choice  paper  and  pencil  test  The 
postlest  was  administered  twicc-oncc  immediately  following  instruction  and  again,  one  week  after  instruction  with  the 

iS&Tpha)" a  *  y  rcm- Rcliabilily  of  lhc  immcdialc  150X110X1  was  -75  and  ***** 

Altitude  Measures.  Pre-  and  post-attitude  questionnaires  were  given  to  assess  student  attitudes  toward  receiving 
instruction  through  cither  cooperative  group  or  individual  deli  very  methods.  The  altitude  questionnaire  was  a  Likcrt-tvre 
•scale  consisting  of  8  .terns.  Reliability  of  the  pre-altilude  instrument  was  .77  and  .79  (o^rJ^S^m 
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Learning  Stvle.  The  Murphy-Mcisgcicr  Type  Indicator  (MMTIC)  (Mcisgeicr  &  Murphy,  1987)  served  as  the 
measure  of  individual  learning  style  preference.  Il  is  based  on  the  same  philosophy  and  learning  theories  as  the  Myers- 
Briggs  Type  Indicator  but  intended  for  second  through  eighth  grade  students.  It  is  a  self-report  instrument  consisting  of 
seventy  items. 

Cooperative  Learning  Check  Test.  The  check  test,  developed  by  the  researcher,  served  to  measure  the  knowledge 
of  cooperative  learning  roles  and  skills.  The  check  lest  consisted  of  ten  yes/no  items  and  had  a  reliability  of  .42. 

Coop  ^tive  Learning  Observation.  A  supplemental  measure  of  cooperation  during  the  study  was  taken.  An 
instrument  developed  by  the  researcher,  to  record  observations  of  cooperative  behaviors  in  groups  was  used.  This  was  an 
eight  item  Likcrt-typc  scale  to  which  the  observer  was  instructed  to  respond  as  a  means  to  gage  a  range  of  cooperation 
from  high  to  low. 

BfisgaEcb  BgsifiD  wti  Analyst 

The  design  of  this  study  is  a  factorial  design  (2  instructional  delivery  conditions  X  3  learning  style  conditions). 
The  independent  variable  of  instructional  delivery  had  two  levels:  cooperative  group  instruction  and  individual 
instruction  For  the  purposes  of  this  study,  cooperative  group  instruction  (CGI)  was  defined  as  three-member  groups 
working  together  implementing  cooperative  learning  strategies,  such  as  positive  interdependence,  facc-to-face  interaction, 
individual  accountability,  interpersonal  and  small  group  skills,  and  group  processing,  to  complete  the  assigned 
instruction  (Johnson  &  Johnson,  1987).  Individual  instruction  (II)  referred  to  one  learner  working  through  the  assigned 
instruction  independently. 

The  second  independent  variable  was  learning  style,  with  three  levels  as  determined  by  the  Murphy-Mcisgcicr 
Typo  Indicator  for  Children  (Mcisgeicr  &  Murphy,  1987).  The  three  learning  styles  were:  extravcrt.  undetermined,  and 
introvert.  This  instrument  provides  cutoff  scores  which  categorize  subjects  as  one  of  these  three  types. 

Both  posttcst  performance  and  attitudinal  data  were  analyzed  with  analysis  of  covariance  techniques  (ANCOV  A) 
using  school  achievement  as  the  covariance.  Support  data  on  the  cooperative  learning  skills  behaviors  (check  test  and 
observation  record)  were  also  analyzed  to  determine  if  skills  were  understood  and  used  by  subjects. 

Experimental  Procedures 

The  classroom  teachers  and  the  computer  lab  teacher  were  taught  cooperative  learning  skills  based  on  the 
Learning  Together  Model  (Johnson  &  Johnson,  1987)  by  the  researcher.  Lecture  and  overheads  were  used  to  present  the 
informal  ion. 

Subjects  were  randomly  assigned  by  the  researcher  to  one  of  six  experimental  conditions,  stratified  by  individual 
learning  style.  The  six  experimental  conditions,  all  receiving  instruction  via  computer  were: 

1 .  Cooperative  Instruction  and  Extravcrt  (CE) 

2.  Individual  Instruction  and  Extravert  (IE) 

3 .  Cooperative  Instruction  and  Undetermined  (CU) 

4 .  Individual  Instruction  and  Undetermined  (IU) 

5 .  Cooperative  Instruction  and  Introvert  (CI) 

6.  Individual  Instruction  and  Introvert  (II) 

Students  assigned  to  CGI  group  were  taught  cooperative  learning  roles  and  skills  by  their  classroom  teachers  and 
given  opportunities  to  practice  them  for  two  weeks,  10  minutes  per  day.  prior  to  attending  instruction  in  the  computer 
lab.  A  cooperative  learning  check  test  was  administered  prior  to  the  study.  A  paper  and  pencil  test,  following  two 
weeks  of  cooperative  learning  instruction,  served  as  a  measure  of  student  knowledge  of  cooperative  roles  and  skills. 

The  majority  of  the  study  took  place  over  a  period  of  three  days  and  in  the  school  computer  lab.  Because  iniact 
classes  were  used,  both  groups  and  individuals  were  required  to  work  at  the  same  time  in  the  computer  lab.  On  Day  1 , 
subjects  were  briefly  introduced  to  the  study.  Cooperative  learning  roles  and  skills  were  reviewed  briefly  for  the  CG 
treatment  groups.  Subjects  received  the  prc-attitudc  questionnaire  and  introduced  to  the  lesson  and  actual  software.  On 
Day  2  after  a  brief  review  of  general  instructions  and  cooperative  learning  skills,  subjects  spent  the  forty-five  minute 
period  interacting  with  the  CBI  software.  The  computer  lab  teacher  monitored  the  students1  behavior  and  assist  them  if 
(hey  had  problems  with  the  computers  (not  the  lesson).  The  researcher  and  other  observer  were  present  to  observe  the 
subjects'  behaviors  in  the  CG  treatments.  On  Day  3,  again  after  the  review,  an  additional  twenty  minutes  of 
instructional  time  was  given.  At  the  end  of  the  CBI  time,  the  immediate  performance  test  was  administered  followed  by 
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tte  pwt-attitude  questionnaire.  One  week  later,  the  delayed  performance  posttcst  was  administered  by  the  classroom 

Summary  of  Results 


Performance 


.  x  3  x  2  analysis  of  covanance  (ANCOVA)  with  repeated  measures  on  the  last  factor  revealed  no 

significant  differences  among  the  six  experimental  conditions  for  performance.  There  was  no  significant  main  effect  for 

TmS  SSSSSS^S repcatcd  mcasures  ^ revcaIed  "°  significant  differcnccs  ^  «* 

Scores  on  the  California  Achievement  Test  (CAT)  Science  sub-section  (covariate)  were  found  to  be  siimificani 
Th.s  .nd>catcs  that  there  was  a  relationship  between  the  covariate  (CAT  scores)  and  performance  g  '' 


Attitude 


Analysis  ot  covanance  (ANCOVA)  revealed  no  significant  differences  among  the  six  experimental  conditions  on 
attitude.  Once  again  mere  was  no  significant  main  effect  A  instructional  delivery,  „£  ^S!S^SSSSSi 
for  learning  style.  In  addition,  no  significant  interaction  was  found  between  the  So  variables.  Sc^  on Sn^uSc 

Support  Data 

Although  no  formal  hypotheses  were  formulated  for  the  following  analyses,  they  were  areas  relarcd  m  ihr 
previous  y  stated  hypotheses  and  served  to  support  or  elaborate  on  the  findings  asSiated  Sh  2  WtaS  £orc  on  the 
CTQPemilvc  hamm^  rlyrk  ir^T  were  above  88%  for  all  six  experimental  conditions.  Tnis  inFonSori SjSSSe 

ZS  E™  3,1(1  hyPOther  ta  1,131  °nC  Ca"  asSumc  *«  sub^  had  «  fairly  good  understanding 

of  the  roles  and  skills  they  were  supposed  to  use  during  the  study.  Tallies  from  nh^Vyations  nf  comaaative  Km 

gmup.  (tallies  of  roles  and  skills  performed  during  interaction  wiihSta^^ 

wdh  an  undetermined  learning  style  exhibited  the  highest  use  of  cooperative  roles  and  skills  fo flowed  to  £»  rtteL 

I?llanh,nlr0VCrtnCarnm8  Sty  ?  "?  fina"y  SubJccts  with  ™  "travert  learning  stv.c.  Diffe  ence  aSSSc 
small,  however  On  an  overall  rating  of  cooperation,  extravcrt  subjects  were  ranked  highest  in  tcr^s  "f  c^Son 
followed  by  undetermined  subjects,  and  finally  introvert  subjects.  coopcrauon, 

rfniiv,™  J"  ?rmary'  ^  f  "l*  °J  Ml Study  failed  t0  suPP°rt  ^  hypotheses  for  interaction  between  instructional 
delivery  and  learning  style  for  both  performance  and  attitude.  This  lack  of  support  may  have  been  due  to  nSoSXical 

IS  o *wr°nC?r  i  ^"n5-  UJ*  ^  dclails  or  limicaitaSof  this  study  wSS^dXSSt 

mtSST^      ngS-  BaSCd  °"  ^  inC°ndUSiVC  TOUlLS'  »"*  ~™dations  for  ^IS^SStSSk^ 

Recommendations  for  Further  Research 
Cooperative  learning  Instruction 

SJKJS,"  £  "TT  Peri°dS'  "°r  WCrC  thcy  promPtcd  «°  usc  «*«  the remSg  S 

St  LZ  *  ,"tcrna,lzc1thcse  cooperative  learning  skills.  Derry  and  Murphy  (1986)  suggest  Uiat  net  skins 

•such  as  these  cooperative  learntng  skills,  be  practiced  and  prompted  for  better  transfer 

It  is  recommended  that  cooperative  learning  techniques  (roles  and  skills)  be  tauaht  to  subjects  nanicinaiino  in  ., 
s  udy.  such  as  was  the  case  in  this  study,  rather  than  assuming  subjects  already  p^^l^^rS^K 

Tvne  of  Cnmpnrpr-Based  Insm^ jnn 
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results  found.  It  is  suggested  that  simulation  or  problem  solving  type  software  be  used,  since  these  types  of  software 
lend  themselves  more  to  discussion  between  group  members  than  does  a  tutorial. 

SSSSded  that  more  subjects  be  included  in  research  such  as  this,  where  subjects  are  identified  and 
assigned  to  ex^ental  clditions  ba  J  on  their  individual  learning  style.  This  is  desired  since  there  ,s  a  strong 
possibility  or  obtaining  disproportionate  numbers  of  learning  style  types. 

Usinn  older  subjects  is  also  suggested  for  farther  research  because  their  learning  style  may  be  more  fully 

subjects  would  increase  the  generalizability  of  the  findings. 

Hnw  the  subiccts  are  grouped  is  another  possible  area  for  further  research.  In  this  study,  by  nature  of  the  design, 

obtained. 

AlSSstudcnus  performed  and  completed  tasks,  their  full  attention  motivation,  and/or  interest  may  not  have 
been  on  th  s  Sn  It  is  befieved  that  these  characteristics  may  have  been  lacking  in  this  study,  due  to  Ae  fact  flat  the 
£££  oSS  so  late  in  the  school  year.  Conducting  research  earlier  in  the  year  may  ehm.nate  this  possible  problem. 

Another  motivational  issue  was  the  incentive  provided  to  make  subjects  think  it  worthwhile  to  Participate  in 
cooncrativcSinrgroups  It  is  recommended  that  the  incentive  be  made  more  appealing,  so  that  group  ™mben> 
wouK  moreTS  to  cl     it  worthwhile  to  participate,  and  therefore  exhibit  more  evidence  or  cooperative  lcammg 
roles  and  skills. 

Recommendations  for  Practical  Application 

^SvIrim^andCBl.  There  arc  still  not  enough  computers  to  make  CBI  an  integral  part  or  classroom 
instructioSfSS^iSechnology  Assessment.  1988;  Hannaf.n.  Dalton.  &  Hooper  1987).  The  use  of 

with  CBI  may  be  one  method  for  solving  the  problem  of  lack  of  computer  equipmen. 

for  students. 

Information  from  this  investigation  may  help  school  personnel  in  the  decision  making  process  regarding 
compute!  S  an! I  Sc  location  of  it.  One  of  the  primary  decisions,  at  the  building  level  is  wheth 
c  ZS TccnS  located,  or  to  house  computers  in  individual  classrooms.  The  results  or  this  study  show  no  d. Terence 
n Srin^cor at  tack  between  students  working  in  groups  or  individually,  ir  it  is  possible  to  aUow  students  more 

T^^X  woLg  in  groups  withinVir  individual Ldj»»n»  J^^S* 
lab  according  to  the  results  or  this  study,  no  significant  decrease  in  performance  or  attitude  shoum  be  expectea. 
Therefore,  when  appropriate,  group  work  may  be  one  w»y  to  increase  student  access  to  computers. 

Lcarnilll^iyJlL  in  terms  or  learning  style,  this  investigation  showed  that  individuals  do  differ  in  terms  or 
....mor  cbSSiksAlthough  no  statistically  significant  interactions  were  found,  this  research  may  serve  to  encourage 
d  S  ^ ^JuSZ  recognize  individual  dirrerences  in  the  classroom.  Once  theft i  dif  erences  are 
rcS^^  make  appropriate  instructional  decisions  responsive  to  the  needs  or  their  particular 

students. 

SSSSai^CBl  The  combination  or  CBI  and  cooperative  group  instruction  offers  a  unique 
opportunity  for  learning.  Along  with  this  opportunity,  comes  implications  for  instructional  design. 
"       For  inswncc,  mere  appears  to  be  a  need  for  software  appropriate  for  group  use.  Much  or  the  mstruc  ion  I 
software  aSfe  was  not  designed  for  group  use.  In  addition,  instructional  design  pr.nc.ples  should  be  appl.ed  to 
'materials  appropriate  Tor  groups,  regardless  or  the  medium. 
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ing^,rt;*^in.gStylf"  *  second  recommendation  is  the  consideration  of  the  target  population  One  steo  in  most 
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Introduction 

The  impact  of  the  "cognitive  revolution"  on  instructional  design  (ID)  has  been 
substantial.  From  the  development  of  ID  models  to  their  application,  instructional  technology 
theorists  continue  to  bring  cognitive  psychology  and  human  information  processing  theory  to 
bear.  The  concepts  and  empirical  findings  of  cognitive  science  are  "helping  to  guide  ID  toward 
new  understandings  of  how  people  learn  and  how  to  design  instruction  for  optimal  results"  (Ely, 
1992,  p.44).  There  is  widespread  agreement  that  each  learner  needs  to  learn  how  to  learn  if  they 
are  to  compete  in  the  technological  world  that  awaits.  So,  in  addition  to  the  dozens  of  other 
instructional  priorities  of  today's  classroom  teacher,  this  author  suggests  that  we  must  also 
instruct  learners  in  metacognitive  strategies:  those  strategies  which  allow  the  learner  10  maintain 
awareness  and  control  of  their  learning,  to  choose  particular  strategies  for  use,  and  to  self- 
monitor  the  use  of  a  variety  of  strategies. 

West,  Farmer  and  Wolff  (1991)  make  a  strong  case  for  embedding  the  instruction  of 
these  strategies  within  the  content.  Research  supports  the  effectiveness  of  mctacognitivc  (raining 
within  varied  content  domains  (Pressley,  Snyder  &  Cariglia-Bull,  1987;  Hansen  &  Pearson 
1983;  Brown,  Campione  &  Day,  1981).  While  these  control  strategics  arc  internal,  learning 
how,  when,  where  and  why  to  use  them  will  be  aided  by  incorporating  their  instruction  with 
content  instruction  (West,  etal,  1991,  p.  1 9). 

Classroom  teachers  arc  typically  not  trained  in  the  practical  applications  of  cognitive 
science  or  human  information  processing  theory.  Most  practicing  teachers  have  had  hide 
exposure  to  instructional  design  principles  or  the  cognitive  theories  which  support  cllcctivc 
instruction.  Nonetheless,  all  teachers  design  instruction  (Briggs,  1977,  p.  179)  at  various  levels 
with  various  expectations  for  the  learner.  The  inscrvicc  described  here  is  designed  (o  teach 
instructional  strategics  that  promote  metacognitive  development  to  these  "front-line" 
instructional  designers.  Fundamental  to  this  inscrvicc  design  is  the  notion  that  instruction  in 
metacognitive  strategics  must  bd  incorporated  into  their  content  instruction. 

Description  of  the  Inscrvicc  • 
Teachers  have  a  strong  sense  of  what  they  want  their  students  to  learn,  but  they  have  a 
weaker  sense  of  how  they  can  most  effectively  help  students  learn,  retain,  and  transfer  their 
knowledge.  This  inscrvicc  design  is  predicated  on  helping  teachers  learn  about  eight  specific 
metacognitive  strategics  and  how  to  embed  strategy  instruction  within  the  instruction  of  content 
objectives.  Strategics  addressed  are:  chunking,  framing,  and  concept  mapping  (organizing 
strategics),  use  of  metaphor  and  use  of  advance  organizers  (bridging  strategics),  rehearsing,  use 
ol  imagery,  and  use  of  mnemonics  (multipurpose  strategies).  This  inscrvicc  is  not  intended  to  be 
a  comprehensive  treatment  of  these  strategics,  nor  an  all-encompassing  exposure  to 
mctacognition,  but  an  introduction  to  effective  metacognitive  strategy  instruction. 

Application  is  directed  at  the  secondary  level  (any  content  area)  but  may  easily  be 
adapted/extended  for  instructors  at  any  level,  K-adult.  Participants  are  expected  to  be  novice  to 
expert  certified  secondary  instructors  and/or  administrative  personnel.  The  inscrvicc  design  and 
companion  reference  manual/workbook  incorporate  the  control  strategies  previously  listed 
Gagnes  nine  events  of  instruction  serve  as  the  defining  methodological  structure  for  the  inscrvicc 
because  most  classroom  teachers  can  relate  to  and  recognize  these  events  as  an  instructional 
model  they  have  implemented  themselves.  The  reference  manual/workbook  is  intended  to  serve 
as  a  personalized  teacher  reference  at  the  conclusion  of  the  inscrvicc;  it  is  not  intended  to  be  a 
stand-alone,  sclf-instniction  module  A  delivery  format  of  two  distinct  one  day  sessions  is 
suggested,  but  might  be  re  configured  to  about  twelve  one  hour  sessions. 

Design  of  the  Inscrvicc 
Participants  begin  their  journey  into  metacognitive  strategics  by  addressing  four 
questions'  What  are  mclacognitivc  strategics?  Why  should  1  teach  mctacognitivc  strategics? 
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When  do  1  teach  metacogniUvc  strategies?  and  How  do  I  teach  mctacognitive  strategics? 
Discussion  of  each  of  these  questions  includes  a  brief  look  at  related  research  and  the  supporting 
cognitive  theory.  Requiring  the  learner  to  be  an  active  participant  in  their  learning  environment, 
motivation,  and  meaningful  learning  arc  examples  of  the  concepts  reviewed.  This  author 
presents  an  eight  step  iterative  instnictional  model  specifically  created  to  provide  teachers  with  a 
structure  for  embedding  metacognitivc  strategy  instruction  within  content  instruction. 

Once  this  conceptual  foundation  is  in  place,  the  inscrvicc  focus  shifts  to  the  eight 
specific  strategies  mentioned  previously.  Content  instruction  for  each  strategy  includes:  a 
definition  of  the  strategy,  a  discussion  of  the  value  of  the  strategy,  examples  of  practical,  cvciyday 
applications  of  the  strategy,  an  activity  related  to  the  classroom  application  of  the  strategy,  and 
modeling  of  the  application  of  the  strategy  within  the  eight-step  instructional  model.  When 
appropriate,  other  theoretical  constructs  are  introduced  (e.g.  types  of  knowledge,  storngc  and 
retrieval)  and  strategies  are  considered  in  combination(s)  for  optimal  effectiveness. 

The  conclusion  of  the  inservice  is  a  discussion  of  factors  which  seem  to  influence  the 
degree  of  effectiveness  of  any  particular  mctacognitive  strategy.  According  to  Snowman  (1986), 
these  factors  are  (1)  the  amount  and  type  of  previous  training  in  the  use  of  the  strategy,  (2)  the 
extent  to  which  the  learner  understands  the  teacher  objectives,  and  (3)  the  ability  of  the  learner  to 
recognize  appropriate  condiiions  for  use.  Helping  students  learn  to  control  their  own  cognitive 
processes  can  have  differential  effects  on  their  future  study  behaviors  and  academic  success.  This 
inscrvicc  strongly  emphasizes  that  teachers  must  help  students  distinguish  between  working 
harder  and  working  longer;  between  working  harder  and  working  smarter  (Weinstcin.  Hagan  & 
Meyer,  1991). 

Results  of  Implementation 

This  author  is  currently  delivering  this  inservice  to  a  group  of  approximately  fifty 
secondary  instructors  from  a  single  high  school  in  northern  Colorado.  Participants  have  the 
option  of  attending  for  university  or  recertification  credit  or  simply  for  the  professional  growth 
opportunity.  Three  participant  are  building  administrators  who  have  a  sincere  interest  in 
increasing  the  quality  of  the  instruction  in  their  school. 

While  no  hard  data  has  yet  been  collected,  the  "reviews*1  are  very  positive.  Attendance 
at  each  successive  session  has  increased.  Participating  teachers  find  this  inservice  professionally 
significant  and  practical;  they  arc  encouraging  their  peers  to  attend.  Teachers  are  "won  over" 
when  they  find  that  instruction  in  mctacognitive  strategics  needs  to  occur  embedded  within  their 
content  instruction,  not  in  addition  to  their  content  instruction.  They  are  enthusiastic  about  the 
concept  of  mctacognilion  instruction  and  the  poicntial  for  creating  an  instructional  environment 
where  accountability  for  education  is  shared;  the  learner  is  responsible  for  controlling  their  own 
learning  and  the  teacher  is  responsible  for  guiding,  supporting,  and  facilitating  that  process. 
Mosi  participants  have  tried  to  introduce  at  least  one  of  the  strategies  using  the  eight  step 
instructional  model.  Within  the  large  group,  they  share  their  successes  and  seek  suggestions  for 
revising  their  Instruction.  The  final  lest  is,  of  course,  whether  or  not  the  majority  embrace 
instruction  In  mcincognillvc  strategics  for  the  long  term  and  adapt  their  daily  instruction  to 
integrate  a  variety  of  the  strategics  presented  in  this  inscrvicc  on  a  regular  basis. 
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Introduction 


Reaearch  into  the  effectiveness  of  practice  strategies  for  learners  has  provided  much 
evidence  that  adjunct  questioning,  a  strategy  where  questions  are  inserted  following  short 
sections  of  information  (Sutliff,  1986),  is  an  effective  educational  tool.  Adjunct  practice 
questions  have  been  found  to  facilitate  learner  recall  and  promote  a  higher  degree  of  learner 
involvement,  even  when  no  feedback  or  opportunity  for  review  are  provided  (Duchaatel  and 
Nungester,  1984;  Watts  and  Anderson,  1971).  Adjunct  practice  questions  have  been  found  to 
^   iUPeri0r  retention  b  bolh  *du,t»  (Hamilton,  1985)  and  children  (Denner  &  Rickards, 
1987),  when  compared  to  reading-only  conditions  with  no  questions. 

Most  studies  involving  adjunct  practice  questions  and  learner  recall  have  examined  the 
effects  of  experimenter-  or  text-provided  questions  (Andre  &  Tbieman,  1988;  Duchaatel  and 
Nungester,  1984;  Hamilton,  1986).  However,  recent  interest  in  the  cognitive  processing  of 
learners  has  produced  a  growing  body  of  literature  about  learners  generating  their  own  practice 
questions  (Cohen,  1983;  Gillespie,  1990).  Studies  of  the  effects  of  learner-generated  questions 
have  found  facilitating  effects  (Denner  and  Rickards,  1987;  Gillespie,  1990;  Steiner,  1988).  In 
theory,  having  students  generate  their  own  practice  questions  should  require  that  they  focus 
greater  attention  on  the  material,  thus  increasing  recall  (Gillespie,  1990).  It  is  hypothesized, 
therefore,  that  learner-generated  adjunct  practice  questions  would  be  more  effective  at 
facilitating  the  recall  of  textual  information  than  experimenter-  or  text-provided  adjunct  practice 
questions. 

Steiner  (1988),  differentiated  between  the  recall  of  lower-order  information  and 
higher-order  information.  Recall  of  lower-order  information  involves  remembering  limple  or 
factual  information  (Hamaker,  1986).  Questions  that  begin  with  'who,"  "what,"  "where,"  and 
when  are  generally  seeking  lower-order  information.  Recalling  higher-order  information 
requires  greater  processing  of  concepts,  procedures,  or  s  synthesis  of  facta,  typified  by  "how" 
and  why  questions.  The  purpose  of  this  study  was  to  investigate  whether  learner-generated 
or  expenroenter-provtded  adjunct  practice  questions  are  superior  at  facilitating  recall  of  lower- 
and  higher-order  textual  information. 
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Method 

fribjocts  ind  Desim 

The  subjects  for  this  experiment  were  74  sixth  grade  students  from  three  classes  k la 
middle  class,  suburban  elementary  school  in  the  southwestern  United  State..  They  embodted 
an  ethnically  mixed  group  of  Caucasian,  hiapanic.  black,  and  asian  studenU.  Subjects  m  each 
class  were  randomly  assigned  to  lower-order  and  bigher-ordar  information  treatment  groups, 
each  containing  an  n  of  37. 

The  design  of  the  study  was  a  2  (authorship  of  practice  question)  X  2  (level  of 
information)  factorial  which  utilized  a  modified  posttest-only  design  with  no  control  group. 
The  dependent  variable  waa  retention  of  information  as  measured  by  performance  on  an 
achievement  postteat. 


Materials 


The  instructional  materials  developed  for  this  study  consisted  of  a  4-page  booklet 
containing  12  paragraphs  of  text,  adapted  from  a  chapter  on  Hinduism  in  the  students  world 
history  textbook.  The  text  material  was  rearranged  so  that  each  paragraph  control  both 
lower-order  and  higher-order  information,  however,  the  baste  content  was  left  unaltered  Six 
of  the  paragraphs  were  each  followed  by  an  experimenter-provided  practice  question  and  a 
space  toanswer  the  question.  The  remaining  six  paragraphs  were  each  followed  bylines 
where  subjects  could  generate  their  own  questions  and  answers.  Although  the  textual  material 
was  identical  for  both  treatments,  the  booklets  given  to  the  lower-order  group  contained  only 
lower-  order  provided  questions  The  higher-order  instructional  materials  contained  higher- 
order  questions.  An  example  of  the  two  types  of  questions  follows: 

Lower-order:      What  are  the  four  classes  in  Hinuism? 

Higher-order:     If  you  were  bora  a  Hindu,  how  would  your  class  decide  your  way  of 
life? 


Criterion  Measure 

The  criterion  measure  was  a  24-item  posttest,  administered  one  day  following  the 
instructional  activity.  Subjectt  did  not  have  an  opportunity  to  review  the  instructional  material 
before  taking  the  teat.  Test  items  included  one  lower-order  and  one  higher-order  item  for  each 
of  the  12  paragraphs  in  the  instructional  booklet.  En  route  data  of  the  time  required  for  each 
subject  to  complete  the  examination  was  also  measured.  Five  studenU  who  were  absent  the 
previoua  dayud  did  not  receive  the  instructional  materials  took  the  posttest  and  were  unable 
fo  aXr  any  of  the  24  test  item,  correctly.  Internal  consistency  of  the  postteat  wa.  calculated 
by  using  the  split-half  reliability  and  Spearman-Brown  prophecy  formula.,  which  yielded  a 
correlation  coefficient  of  .83. 
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Procedural 


oe„.«»i^PPfT        Vf?  month  prior  to  the  itudy,  the  .ubjecU  received  instruction  in  the 

fiT  "S  IU-Uom  by  their  classroom  teacher.  He  teacher 

differentiated  b*weeo  "factual"  (lower-order)  and  "conceptual"  (higher^rder)  queetioning 

given  opportunities  to  practice  questioning  .kills  in  class  sod  on  homework  alignments. 

,  *fore  «e  instructional  materials,  the  subjects  were  given  a  brief  review 

in  the  difference  between  "factual"  and  "conceptual"  questions.  Subjects  in  the  lower-ordeT 
question  treatment  were  directed  to  compose  lower-order  questions  while  objects  fa,  the 
higfaerorder  treatment  were  directed  to  compcae  higher-order  questions.  The  subject,  were 
directeo  to  write  answers  for  all  12  questions.  J 


Results 


.       J?T  ?*^  J  «Penmenter-provided  and  learner-  generated  practice  questions  are 
shown  in  Table  1.  Data  for  porttert  achievement  were  analyzed  using  a  2 X  2  factorial 
analy»s  of  variance.  One-  and  two-way  analysis  of  variance  showed  a  significant  effect  for 
auftorship  of  adjunct  practice  questions  at  both  lower  F(l,72)  -  8.87,  p<  .01,  MSe  -  2  20 
snd  higher  F(l,72)  -  10.59,  p<.01,  MSe  -  2.47  ^formation  levels.  The  tnJn t,me  o^the 
expenmenter-prov.ded  group  were  found  to  be  significantly  higher  than  those  for  the  Jamer 
grated  group.  No  significant  effect,  for  level  of  practice  question  Jgtder  w.„ 
reported.  No  ..gmficant  mteraction  effecU  were  reported.  These  remits  fail  to  support  the 
hypothes..  that  geoerstmg  adjunct  practice  question,  would  be  superior  at  facilitatwTrecall 
when  compared  to  experimenter-provided  practice  questions.  ' 

Subjects  in  the  lower-order  group  took  an  average  of  5.48  minutes  to  answer  the 
^k^ave^lTf  10  gMer,te  ^ti0M-  "      higher-order  group 

question]  °m      10  qU*1,ion,  ud  to**  12.03  minutes  to  generate 

TABLE  1:  Recall  by  Level  snd  Authorahip  of  Adjunct  Practice  Questions 

4uthoxsbip 


Level  of  Information  Provided 


Lower-Order 

M 

3.92 

2.89 

6.81 

SD 

1.57 

1.39 

2.84 

Higher-Order 

M 

4.03 

2.84 

6.87 

S 

1.38 

1.57 

2.65 

Total  M  7.95 


5.73  13.68 
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Diacuuion 

Both  provided  and  generated  adjunct  practice  question,  have  been  ehowo  to  be  an 
effective  aid  in  facilitating  recall  of  text  information  when  compared  with  a  reading-only 
condition..  BecauM  of  the  lncre«ed  cognitive  activity  required  to  generate  qi^ion..  It  was 
hypothesis  that  sixth  grade  learner,  who  generated  their  own  practice  queatlon.  would  snjoy 
superior  recall  than  when  they  angered  provided  practice  questions.  The  finding,  of  thU 
study,  however,  lupport  the  mperiority  of  provided  quertiona  over  generated  quMtiona  for 
SgnSer..  TnUfbding  i.^onsi.teot  with  Denner  and  Rickaid.  (1987),  who  found  that 
fifth  grader,  recalled  subordinate  and  superordinate  dcUil.  better  with  provided  than  with 
aenerated  questions.  Thi.  finding,  however,  was  reverted  with  eighth  grader..  Tnis  result 
I oiw  bedue  to  the  fact  that  .ixth  grtder.  are  more  like  fifth  grtder.,  who  may  not  yet  pooea. 
the  cognitive  skill,  neceeaary  to  for  the  taak  of  generating  and  encoding  interrogatory 
information. 

Theae  rteulU  may  be  due  to  a  high  correlation  which  existed  between  the  provided 
question,  and  the  item,  on  the  posttest.  Students  who  generate  practice  queation.  dissimilar  to 
tLe  on  the  teat  will  be  working  under  a  handicap,  rather  than  an  adv«ta«eJThi.  handicap 
can  be  lewened  greatly  by  providing  learner,  with  opportunities  to  receive  feedback  regarding 
the  quality  of  their  question.,  sod  the  chance  to  revise  question,  prior  to  utihang  them  for 
recall  practice.  The  desirability  of  maintaining  a  learner-generated  adjunct  questioning 
strategy,  including  extra  time  for  feedback  and  revision,  i.  an  isnie  that  must  be  decided  upon 
by  the  individual  instructor. 

For  .ixth  grade  learner.,  providing  q  lestions  wear,  to  afford  more  effective  and 
efficient  recall  than  having  the  learners  generate  them.  Since  adjunct  practice  questions  have 
teen  tucce..fiil  a.  a  strategy  for  learner  recall  of  text  information,  their  continued  uuge  a.  sn 
imtructiotul  strategy  it  recommended. 
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on^,.AJt5°U9h,a  ,reat  deal  ?£  "search  has  been 
conducted  on  learner  control  and  on  feedback,  few 

linfi!!?  !!aVe  been  condufted  t0  determine  if  leaner 
ZZuaIZI       £eedback  have  ■  beneficial  effect  on 

student  performance  and  attitude.    According  to 
Clanana,  Ross,  and  Morrison  (1991)  feedback  is  an 

SKliS MlfXabi*  th?fc  X!  o£t,n  ^ond  in  computer 
assisted  instruction  (CAI).    Yet,  it  is  possible  to 

fmoin?  i?8JrU^i0u  that  allowa  to  control  the 

amount  of  feedback  after  a  practice  item.    Using  a 

«™«r  t0  •dminj8t«r  feedback  can  be  an  efficient 
IIZImI'  /  «?«»ut.r  can  act  as  a  sophisticated  tutor, 
capable  of  adjusting  feedback  to  obtain  effective  and 
efficient  instruction  (Anderson,  Boyle  6  Reiser,  1985). 

nontro?  f£*?i0U?  stud*'  u«d  Program  and  learner 
Jl!h«it?i«?  tw°  levels  °f  feedback  (verification  and 
elaboration).    These  are  the  two  components  of  feedback 

tST^  byKKUl5"V£  6  St0ck  ("99).  The  results  of 
that  study  showed  that  elaboration  was  significantly 
more  influential  than  verification  in  producing  greater 
performance      in  this  study,  we  wanted  to         if  the 
degree  of  elaboration  feedback  needed  for  high 
performance  could  be  determined.    We  used  program  and 

ve^?ica^oir01        thre*  leVClS  °£  which  Sere 

verification,  correct  answer  and  elaboration. 


Method 


Subjects 

at  e  ?S«ICiJ,.?!!e  I26  und^«r«duate  Education  majors 
Jh-f  liJl  ;outh?«»Urn  university.    They  were  told 

telchlrl  inl0^^  Would  b«  u"«"l  to  them  a.  future 
teacners  and  that  they  would  receive  credit  for 
participation  in  the  study. 

Procedures 

k„i    ThlS  5lUdy  WM  ■dminietered  by  microcomputer  and 
had  six  conditions  with  21  subjects  in  each  group  All 
subjects  read  text,  answered  embedded  practiSS 

32rtXS!MllJ  received  Redback.    They  then  completed  a 
short  attitude  survey  and  a  posttest  consisting  of  the 
practice  questions  presented  in  random  order 

Before  subjects  arrived  to  participate  in  the 
study    we  prepared  the  computer  laboratory  by 
£H  li1?9  one  °f  the  six  lessons  into  each  computer. 
Upon  arrival  at  the  computer  room,  subjects  were 
SX?25l  asai*nad  t0  ««ch  °f  the  six  conditio^ 
?«^?f-S  Were  run  in  «r0UP«  of  about  25  with  all 
conditions  present  at  each  session. 

.  #i  i."?^*76  ■  short  introduction  on  general  procedures 
and  told  subjects  that  instruction.  wirS  "cluleS  In 
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the  program.     Thty  were  not  told  that  the  programs  were 
different.     Subject!  were  told  that  the  lesson  was  on 
reliability  and  validity  and  stressed  the  importance  of 
".material  for  them  as  future  teachers.  Subjects 
then  proceeded  with  their  individual  lessons.  Upon 
completion  of  the  lesson,  each  subject  completed  the 
attitude  questionnaire  and  the  posttest  on  the 
computer.    Subjects  were  given  "  nrnch  time  a  they 
needed  to  complete  the  lessons  and  the  criterion 
measure.     So.t  subject,  completed  the  .tudy  within  a 

50-minute  class  psriod.  ,     . _ 

The  difference,  in  treatments  occurred  in  the  type 

of  control  (program  or  learner)  and  Ijvel  of 
(verification,  correct  answer,  or  elaboration).  Under 
the  condition  of  program  control/verification,  a 
eSpSSr  Program  delivered  feedback  telling  subjects 
that  their  answer  was  correct  or  incorrect. 
program  control /correct  answer,  the  feedback  delivered 
by  the  computer  was  whether  or  not  the  response  was 
correct  and  the  correct  answer  appeared  on  the  screen. 
Under  program  control/elaboration,  the  computer  showed 
subjects  whether  their  response  was  correct,  tne 
correct  answer,  and  a  short  explanation. 

The  three  learner  control  conditions  contained  the 
same  levels  of  feedback  as  program  control  conditions, 
but  only  appeared  when  learners  chose  to  see  the 
feedback  for  their  condition.     After  learners  responded 
to  a  practice  question,  they  were  asked  one  of  three 
sets  of  questions  as  follows:     1)    For  the  verification 
group  -  Would  you  like  to  check  you  answer?      2)  For 
the  correct  answer  group  -  Would  you  like  to  check  you 
answer?    Followed  by  -  Would  you  like  to  see  the 
correct  answer?    Or  3)  For  the  elaboration  group  - 
Would  you  like  to  check  your  answer?    Followed  by  - 
would  you  like  to  see  an  explanation? 

MateCMaterials  used  in  this  study  were  six  computer 
programs,  a  posttest,  and  an  attitude  *u;B"~nf"*:__t 
The  computer  based  lessons  were  developed  from  the  text 
L  "  in !5LsuremfiBtj  Reliability  and  validity  by  Dick 
6  Hagerty  (1971).     Information  and  examples  were 
presented  in  sections  of  five  screens  of  text,  followed 
by  eight,  five-alternative,  multiple-choice  questions. 
This  cycle  continued  for  a  total  of  25  screens  of  text 
and  40  questions.  , 
Differences  in  the  lessons  were  based  on  type  or 
control  (program  or  learner)  and  level  of  feedback 
(verification,  correct  answer,  or  elaboration). 
Program  control  groups  received  one  of  the  levels  or 
feedback  automatically.     Learner  control  groups 
received  verification,  or  correct  answer,  or 


805  <>0o 


elaboration  when  they  requested  feedback.  Because 
verification  was  given  to  program  control  groups  at  all 

\-MrMrS  With  contr°l  ^  the  correct  answer  and 
elaboration  groups  had  to  choose  verification  before 

^•?ha?Ucondnron.t0         ^  ^  °f  feedba£*  assi«n«d 

r,«-ffin4.*ddition  to  the  six  comPUter  based  lessens,  a 
posttest  and  an  attitude  questionnaire  were  developed 
Th.  posttest  consisted  of  the  same  40  questions 
previously  given  as  practice  but  presented  in  random 
°Jd*l«    T?*  ty  of  the  posttest  was  c.lcSl^ed 

Iff4fL!*in9  J?*  Kud«r-Richardson  20  formula.  The 
Itudlni  .2??I^°?faira  consi9ted  o£  ten  items  measuring 
J dSiii;i *!  Uct*on'  «Ooy»«nt,  perception  of  control! 

tiv.  !!Ji?9nfWafdt£Mdback-    The  W«tionn.ir«  used  a 
^«!n?Jln    5ik«rt:tyP«  scale.     Both  measures  were 
administered  on  the  computer,     in  addition,  the 

^ha!!w/UJ°mati?ally  rec°rded  the  number  of  seconds 
each  subject  spent  studying  feedback  messages. 

^n^nheMeSi9n  was  a  2  x  3  factorial  with  type  of 
control   (learner  or  program)  and  level  of  feedback 
In^i:1^*  l°n'  Co5feot  answer,  or  elaboration)  as  the 
independent  variables.     The  dependent  variables  were 
performance,  attitude,  and  feedback  study  time. 

Resul ts 

«„.«.4.Retentior\as  measured  by  performance  on  the 

The  mean  llllll**  the  influence  °*  type  of  feedback. 
Tfte  mean  scores  for  program  control  subjects  on  the 
posttest  were  21.4,  29.7  and  30.8  for  verification 

TCLrrsetanSSdeHr  andfelab°"tion  « -dteek'^ti vei y . 
4  8  deviations  associated  with  each  mean  wSre 

indiJ;L  ?k  ?-\resPe5tlvely-     The  "lean  scores 
answer  it  11V  fuh^cta  *iven  the  correct 

a2r?nL!J  Jh«  corr«ot  answer  with  elaboration,  subjects 
performed  better  on  a  posttest  than  if  they  were  only 

?;™rerifiCati°nv    A  Similar  P»ttern  occXr?ed  for  Y 
learner  control  subjects. 

f.       Th*  mean  scores  for  learner  control  subjects  on 
the  posttest  were  20.3,   23.7  and  25.0  for  verification 
correct  answer  and  elaboration  feedback  respectively 
I  2  de*iations  associated  with  each  mean  were 

pattern  respecti vel * ■     Here  again  the  same 

of  «I2h!rt  1;creased  P«f°rmance  with  increased  amount 
of  feedback  emerges,  but  not  as  strongly  as  for  oroaram 

re^db-ik81113?6^3^    SubjCCts  wh°  rec^d  "rifircJtiS"m 
rlnt^    J  °nll  dld  not  Perf°™  as  well  as  subjects  who 
SrtwS  11    "  cor"ct  answer  or  elaboration  feedback, 
best  wni^ir  5fCeiVed  elabor«tion  feedback  performed 
best  whether  they  were  program  or  learner  control 
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subjects.     Program  control  subjects  performed  better  on 
the  posttest  than  learner  control  subjects  at 
comparable  feedback  levels. 

An  ANOVA  analysis  revealed  significant  main 
effects  for  scores  on  the  posttests  with  type  of 
control  at  F( 5 , 120)=19. 67 ,  r<.05  liSe=29.42  and  level  of 
feedback  at  £<5,120)=20.17,  B<.05  H£e=29.42.  However 
the  interaction  did  not  reach  the  standard  .05  level  of 
significance.     The  interaction  was  significant  at  the 
.06  level . 

The  results  of  feedback  study  time  showed  that 
program  control  subjects  spent  more  time  studying 
feedback  than  learner  control  subjects.  Program 
control  subjects  averaged  44.2,  101.7  and  232.0  seconds 
respectively  for  verification,  correct  answer  and 
elaboration  conditions.     However,  learner  control 
subjects  spent  26.5,  37.4  and  82.1  seconds  for 
equivalent  feedback  conditions.     An  ANOVA  analysis  of 
feedback  study  time  revealed  a  significant  interaction 
F(2,120)=22.46,  &<.001  HSe=2101 . 90 . 

To  analyze  the  attitude  survey,  a  MANONA  analysis 
was  first  run  to  see  if  there  were  significant 
differences  between  the  groups  on  all  questions 
combined.     Results  revealed  a  significant  MANOVA  effect 
for  level  of  feedback  £( 20 , 100)  =  2 . 12 ,  £<.05.  Follow-up 
univariate  analysis  showed  two  questions  to  be  highly 
significant.     The  one  question  was  "1  would  have  liked 
more  control  over  the  lesson"  F( 2 , 120) =5 . 31 ,  p<.01. 
The  other  question  was  "I  would  have  liked  more 
feedback  about  my  answers"  F( 2, 120)  =  13. 90  ,  £<.01. 

Discussion 

This  study  and  the  original  study  both 
demonstrated  that  verification  feedback  alone  was 
insufficient  for  subjects  to  reach  high  levels  of 
performance  on  a  posttest.     This  study  showed  only  a 
minimum  improvement  (one  question  on  the  forty-question 
posttest)  when  elaboration  was  added  to  the  correct 
answer  feedback.     The  additional  time  and  expense 
needed  to  design  the  instruction  with  elaboration 
feedback  is  probably  not  warranted.     Students  would 
perform  at  almost  the  same  levels  when  only  given  the 
correct  answers. 

The  pattern  of  improved  performance  from 
verification  to  correct  answer  to  elaboration  was  seen 
in  both  program  and  learner  control  conditions. 
Program  control  subjects  always  outperformed  learner 
control  subjects.     This  would  lead  one  to  recommend 
program  over  learner  control.     It  should  be  mentioned, 
however,  that  there  were  greater  differences  in  the 
standard  deviations  for  learner  control  correct  answer 
and  elaboration  conditions  (7.5  and  6.4)  than  for  the 
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other  four  conditions.     Those  conditions  ranged  from 
J.B  to  4.8.     The  larger  deviations  were  a  result  of 
JI-JE!: w lea5n*r.Contro1  subjects  choosing  very  little 
teedback  and  doing  poorly  on  the  posttest  but  not 
poorly  enough  to  be  considered  outliers  that  could  be 
dropped  from  the  analysis. 

The  results  for  the  feedback  study  time  were  as 

!?SJv,^'f.L!Krn!r.u°ntro1  sut,3ects  spent  less  time 
studying  feedback  than  program  control  subjects.  When 
subjects  were  given  the  opportunity  to  skip  feedback, 

i„!ffi!?      c  dt\S°'    This  resulted  in  lower  scores  on  a 
posttest.     Sometimes  lower  scores  are  offset  by 
improved  attitude  toward  the  instruction.     That  was  not 
the  case  with  the  current  study. 

No  differences  were  found  between  types  of  control 

<  J1*??6  ,Study-     The  onl*  differences  were  for 
level  of  feedback.     Subjects  who  were  only  given 

«Jri«i?J  i°n  !?efe  universally  displeased  with  the  lack 
of  control  and  lack  of  feedback. 

This  study  would  suggest  that  program  control  with 
correct  answer  feedback  is  the  most  effective  and 
efficient  way  to  present  instruction  on  a  computer. 
However,  additional  research  with  different  subject 
matter  and  different  age  subjects  would  be  warranted. 
More  sophisticated  programs  could  also  be  developed 
that  would  only  allow  learners  to  omit  feedback  when 
they  have  given  correct  answers.     That  situation  may 
keep  scores  high  and  improve  attitude. 
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An  Assessment  of  the  Effectiveness  of  a  Hypertext 
Instructional  Delivery  System  when  Compared  to  a 
Traditional  CAl  Tutorial 

Ann  M.  Quad* 
Department  of  Computer  and  Information  Sciences 
Mankato  State  University 
Mankato,  MN 

More  than  40  years  ago,  Vannevar  Bush,  science  advisor  to  President  Roosevelt, 
proposed  a  hypothetical  machine,  the  MEM IX,  that  would  allow  Individuals  to  browse 
through  information,  photographs,  maps,  and  sketches  by  using  links  connecting  any  two 
source  documents.  The  visionary  image  provided  by  Bush  coupled  with  significant 
advances  in  computers,  software,  and  high  capacity  fast  access  digital  storage  devices 
have  given  rise  to  databases  that  today  contain  documents,  graphics,  sound,  speech 
and  animated  sequences.  A  database  containing  information  In  these  varied  formats 
which  supports  the  non-sequential  access  of  information  by  the  user  has  been  coined 
hypertext. 

Hypertext  can  be  visualized  as  a  "semantic  netwoik  of  nodes  and  links.  In  this 
model,  nodes  represent  information  units  (e.g.  documents,  graphics,  sound,  etc.)  while 
links  serve  as  the  cross-reference  threads  connecting  related  nodes.  The  links 
connecting  nodes  are  usually  denoted  by  highlighted  words,  phrases  or  graphics  and  are 
activated  by  using  a  mouse  or  arrow  keys  to  select  It.  Depending  on  its  purpose  and 
location,  a  link  can:  transfer  the  reader  to  a  new  but  related  topic,  show  a  reference  or 
serve  to  move  from  a  reference  to  the  article,  provide  ancillary  information,  display  a 
related  illustration,  schematic,  photograph  or  video  sequence,  display  an  index  or  access 
and  run  a  related  application  program. 

The  development  of  hypertext  systems  has  grown  rapidly  during  the  last  decade. 
Today,  applications  which  utilize  hypertext  include:  online  help  and  documentation 
resources,  software  engineering  tools,  encyclopedias,  reference  manuals  and  books 
and  computer  assisted  learning  in  numerous  educational  disciplines. 

Hypertext  has  three  characteristics  that  are  not  present  in  traditional  instruction 
which  relies  heavily  on  a  highly  directed  flow  of  information.  First,  hypertext  systems  allow 
huge  collections  of  information  in  a  variety  of  mediums  to  be  stored  in  an  extremely 
compact  form  and I  accessed  easily  and  readily.  The  stored  information  can  be  linked  both 
explicitly  and  implicitly.  Explicit  links  can  be  used  by  authors  to  suggest  paths  through 
information  which  learners  may  or  may  not  choose  to  follow.  Implicit  links  can  be  included 
which  support  glossaries,  dictionaries,  and  navigational  aids  to  be  used  as  needed  by  the 
Itamer  (Marchionini,  1 988).  y 

Second,  hypertext  is  an  enabling  rather  than  directive  environment.  Hypertext 
systems  encourage  the  user  to  probe  Information  sources.  By  selecting  the  order 
number  and  composition  of  the  nodes  accessed,  one  can  acquire  and  process 
SffiSS?" in  way«^?t  caP|,allze  on  tnelr  Individual  cognitive  and  experiential  framework 
Scnneiderman,  1990).  Learning  theorists  contend  user  Initiated  movement  also 
ncreases  the  number  of  retrieval  pathways  available  to  access  encoded  Information  from 
the  user's  own  long  term  memory  (Jonassen  ,1988).  Not  only  does  hypertext  offer  a  new 
way  to  learn  course  content  but  also  offers  a  new  way  of  learning  how  to  learn. 

.  ,  ThJfd-  hypertext  offers  the  potential  to  after  the  roles  of  teachers  and  learners  and 
the  Interactions  between  them.  The  flexibility  of  hypertext  enables  students  to  create 
unkiue  paths  and  interpretations  of  the  paths  which  later  can  be  shared  with  teachers  or 
other  students  (Marchionini,  1988). 

.u     C™^861*  0,hers  ar0uo  ,nat  hypertext  Is  less  than  optimal.  Learner  control, 
aptitude-treatment  interaction  (ATI),  disorientation  and  cognitive  overhead  are  problems 
often  associated  with  hypertext. 

,  Fo  •  somS  learners.  the  degree  of  control  afforded  by  hypertext  may  negatively 
affect  learning.  Tennyson  and  Rothen's  (1979)  research  on  learner,  adaptive,  and 
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Droaram  control  strategies  employed  in  computer  based  instruction  concluded  that 
fea^cSffconsiZtly  yielded  lower  post-test  scores  than  other  co ntro  strateo'es 
Dart  ally  because  students  terminated  instruction  too  early.  Because  of  the  richness  of 
fSJSSnSSnm  may  not  know  what  sequence  or  content  ^SUffSSiST 
or  themselves  to  reach  desired  learning  objectives.  Some  suggest  that  the itess  famlHar 
subieds  are  wiE  the  content  of  the  unit  to  be  mastered,  the  greater  the  need  or  support 
in  the  form  of  clearly  stated  learner  objectives.  Such  objectives  provided  to  the  learner 
could  serve  as  a  'checklist1  indicating  to  the  learner  what  must  be  learned  while  using  the 
hypertextsystem.  Objectives  used  in  this  manner  provide  additional  guidance  and 
direction  as  learners  utilize  hypertext  for  instruction. 

Other  issues  related  to  instructional  control  appear  to  be  directly  related  to  the 
learner  himself.  Hannaf  in  (1984)  concluded  from  reviewing  relevant  research  that  learner 
control  is  favored  over  program  control  when:  (1 )  learners  are  older  and  more 'mature, ,  (2) 
learne  s  are  more  capable;  (3)  and  higher  order  skills  rather  than  factual  Information  are 
b^lng  tLght  Stated  another  way,  hypertext  may  not  be  the  best  mode  of  instruction  for 

all  learn®^t^[J^"trol3tmen?interactIo n  phenomenon  may  be  also  be i  aggravated  by  the 
lack  of  structure  In  such  a  learning  environment.  Research  by  Claik  (1982)  indites  tow- 
ability  students  have  higher  achievement  from  more  structured  Instmction^ 
whereas  they  prefer  less  structured  methods.  On  the  other  hand,  ^jh-abHrty  stents, 
he  claims,  wHI  generally  prefer  more  instructional  support  but  may  actually  pe*oim  &jtter 
when  given  leaner,  less  elaborated  material  where  they  must  take  more  responsibility 'tor 
aenera inc (encoding  strategies.  Lower  ability  students,  who  need  the  additional  support, 
mTy  deSSe  Hek  rt.  He  hypothesized  that  both  groups  prefer  what  they  perceive  will 
demand  the  lowest  "mental  workload"  for  themselves. 

Additionally,  Steinberg  (1977)  addressed  the  issue  of  ATI.  ^e  suggested  that 
while  high  achievers  seem  capable  of  using  most  forms  of  learner  control  etfectlvely  tow 
acWevers  seem  much  less  able  to  make  decisions  about  instructional  properties  (what, 
how,  or  how  much  information)  than  from  those  decisions  involving  variations  in 
presentational  aspects  (how  information  is  formatted  or  delivered).  Her  research  flings 
suggest  when  Instructional  decisions  need  to  be  made,  better  learning  Is s  Ilk ely  to  occur 
with  external  coaching  or  advisement  regarding  which  resources  to  select  Aptitude- 
treatment  interaction  research  suggests  hypertext  may  not  be  the  most  •™ct!ve 
environment  for  low  achieving  learners.  Perhaps  a  strategy  which  prescribes  closer 
assessment  of  learning  by  the  instructor  may  be  necessary.  ...... 

Disorientation  is  the  result  of  more  degrees  of  freedom,  more  dimensions 
available  in  which  one  can  move.  Technical  solutions  to  dlsorientatton  are:  (1 1  fljjhteal. 
map-like  models  which  depict  the  current  hyperspace  and  identify  the  users  position 
within  the  space;  (2)  and  providing  a  means  for  the  learner  to  backtrack. 

Graphical  models  termed  context  webs  and  local  maps  are  used  In  Brown 
University'slntermedla  system.  In  this  hypertext  system,  nodes  consisting  of  documents, 
graphics,  sound,  and  animation  related  to  the  same  context  are  grouped  to  form  a  web. 
Using  this  model,  every  node  belongs  to  one  or  more  web.  When  the  user  activates  a 
node  belonging  to  a  web,  they  can  visually  see  the  other  links  and  nodes  which  also 
belong  tc i  thrive  web.  In  addition,  a  local  map  can  be  activated I  whtoh  only  shows  the 
present  active  document  and  Its  closely  related  neighbor  nodes  (Conklln,  1987). 

HyperCaid  and  other  developmental  shells  offer  a  back-tracking  function.  By 
clicking  on  an  icon  representing  a  prior  screen,  the  user  is  able  to  move  directly  to  that 
screen  thus  enabling  a  rapid  review  ol  previously  encountered  information. 

The  additional  mental  effort  and  concentration  required  to  make  choices  about 
which  paths  to  follow  and  which  to  leave  alone  can  result  In  'information  myopia  or 
cognitive  overhead  (Conklln,  1987).  Cognitive  overhead  may  be  reduced  if  the  user  is 
able  to  more  effectively  organize  and  Integrate  the  new  chunks  of  information  extracted 
from  the  hypertext  medium  with  existing  prior  knowledge. 

To  date,  little  research  has  been  conducted  to  support  or  resolve  the  above 
mentioned  issues  related  to  hypertext  systems.  Research  performed  by  Egan,  et.  ai. 
M9871  and  Landow  (1989)  support  the  contention  that  hypertext  systems  do  in  tact 
support l  l I  richer  learning  e^onment.  Other  studies  conducted  by  Covey  (1 990),  Lanza 
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intellectual  property  (copyritt^  ^  X° 


Subjects  Method 

midwestern  state  university.  The  subjects  LriXfn ?J£ 8fc?°l  efle  °f  Business  at  a 
group  received  instruction nusina  a  CBI SK1?JSd.,2,°  *"°  treatmen«  Groups:  one 
delivered  in  a  hypertext  format  Shich  ii  22  °,her  W  received  ins<™tk>n 

Programs 

proper  o^oTfiSl'  ^1^^,23^  Zfij""  «-  toptes  in  the 
to  identifier  Ke^K  **  made  no  attempt 

Fac/W/es 
Instruments 
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presented  In  the  software.  One  point  was  allocated  to  each  question.  This  instrument 
was  used  as  both  the  pre-test  and  post-test  assessment  instrument. 

The  reliability  of  this  Instrument  as  determined  by  a  KR-20  analysis  was  .688. 

Procedure 

The  procedure  used  In  this  study  was  as  follows: 

1 )  Two  days  prior  to  the  study,  subjects  unfamiliar  with  the  Macintosh  platform  used 
to  deliver  both  forms  of  instruction  received  an  intensive  two  hour  training 
session.  M  , 

2)  Treatments  were  assigned  randomly.  Detailed  instructions  on  effective  use  of 
the  assigned  treatment  as  well  as  a  practice  period  using  software  similar  to  the 
assigned  treatment  was  implemented  for  each  participant. 

3)  After  the  practice  period,  subjects  were  administered  a  pre-test  and  given 
objectives  reflecting  the  subject  material  to  be  mastered. 

4)  Participants  were  administered  a  treatment  consisting  of  either  the  tutonal  or 
hypertext  system  and  post-test  under  supervision  of  the  researcher.  No  time  limit 
was  placed  on  viewing  the  software. 

Results 

Table  1  reports  the  t-test  analysis  which  was  applied  to  the  results  of  the  post-test 
to  assess  the  difference  in  performance  between  the  tutorial  and  hypertext  treatments. 
No  significant  difference  was  found  at  the  ,05  level. 


Table  1.  Tutorial  vs.  Hypertext  Group  Post-Test  Results 


N 

Mean 

S.D. 

Tutorial  group 

37 

20.158 

3.6865 

Hypertext  group 

39 

18.313 

3.6160 

t(74)  d(-.45  p-.643 
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Table  2.  TutorlaJ  vs.  HvDflrtflxt  r,rn„r  paired  Pre-Test.  P^jest  Results 


N 

Mean 

S.D. 

Tutorial  group 

Pre-test 

Post-test 
Hypertext  group 

Pre-test 

Post-test  

37 
37 

39 
39 

12.4054 
18.5405 

13.0513 
18.8205 

2.544 
4.018 

3.268 
3.546 

.029 


ability  and  medium  ability  groups      &,8ni,,cance  was  found  at  the  .05  level  for  both  low 


Table  3.  Ability  Groups  Paired  Priest.  Post-Test  R-„it« 


Low  ability 

_  P 

Pre-test 
Post-test 
Medium  ability 

21 
21 

11.8571 
17.0000 

3.321 
3.987 

.036 

Pre-test 
Post-test 

36 
36 

12.7500 
18.1389 

2.951 
3.217 

.030 

High  ability 

Pre-test 
Post-test 

19 
19 

13.6642 
21.5789 

2.212 
2.893 

.902 
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A  one-way  analysis  of  variance  (Table  4)  showed  no  signifteance  at  the  05  level 
between  ability  and  the  pre  test  scores  but  significance  at  the  .05  level  between  ability 
and  the  post-test  scores.  Further  follow  up  with  the  Scheffe  procedure  indicated  abiMy 
groups  one  and  two  (tow  and  medium  ability)  and  groups  two  and  three  (medium  and  high 
ability)  were  significantly  different  at  the  .05  level. 

Table  4.  One  Way  Analysis  of  Variance  Pre  -Test.  Post-test  by  Ability 

variable     Source  of  Variation     d<  SS  MS_ 


Pretest 

Between  Group* 

2 

33.3102 

16.6551 

1.982 

.1451 

Within  Groups 

73 

616.4267 

8.4031 

Post-test 

Bstwttn  Groups 

2 

229.4839 

114.7420 

10.081 

.0001 

Within  Groups 

73 

630.9371 

11.3827 

The  data  associated  with.screens  viewed  per  treatment  is  summarized  in  Table  5. 
No  significant  difference  between  means  was  found  at  the  .05  level. 


Table  5.  Tutorial,  Hypertast  Screens  Viewed 
Tutorial 

Hypertext  


A  one  way  analysis  of  variance  (Table  6)  showed  significance  at  the  .05  level 
between  the  post-test  tutorial  scores  and  ability  groupings  while  no  significance  was 
found  between  post-test  hypertext  scores  and  ability  groups. 


N 

Mean 

SD 

37 

123.824 

26.903 

39 

118.324 

15.813 

Dsosndsnt  variable     Source  of  Variation 

df          SB  MS 

F 

Tutorial  trsatmsnt     Bstwssn  groups 
Within  groups 

Hypertext  treatment  Between  groups 

Within  groups 

2     210.6237  105.3118 
34    376.1871  11.0643 

2      31.2526  15.6263 
36     507.3000  14.4943 

9.5181 
1.0781 

.0005 
.3513 

Discussion 

The  purpose  of  this  study  was  to  assess,  in  an  undergraduate  university  setting, 
the  effectiveness  of  a  hypertext  instructional  delivery  system  when  compared  to  a 
traditional  CAI  tutorial.  Several  findings  which  relate  directly  to  problematic  issues  of 
learner  control  and  ability  were  noted.   ,  . 

Learner  control  over  the  number  of  Informational  screens  viewed  was  not  found 
to  be  a  factor  in  overal  subject  performance.  Quantitative  examination  of  the  audit  trail 
compiled  by  each  subject's  treatment  indicated  there  was  no  significant  difference  in  the 
number  of  subject  matter  screens  viewed  by  the  tutorial  versus  hypertext  treatment 
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groups^  Subjects  in  both  treatment  groups  appear  to  have  taken  the  treatments 
eamestry.  Each  treatment  group  viewed  nearly  one  fourth  of  subject  screens  twice  The 
novelty  of  the  instructional  mode  should  not  be  ruled  out  in  further  studies  as  a  factor 
which  may  play  a  significant  role  in  this  aspect  of  evaluating  learner  control. 

Although  learner  control  in  this  study  did  not  hinder  a  subject's  ability  to  navfaate 
through  the  screens,  it  did  not  facilitate  a  greater  understanding  of  the  material  as 
assessed  by  the  post-test  Instrument.  Neither  treatment  was  found  to  achieve  a  greater 
measure  of  learning  of  intellectual  property  law.  « 

aH^JUl?  ? eir!?iSat,,ons  ,rom  tnis  study that  a"111*  Pty* a  maK>r  role  in  assessing  the 
fj^a^ofa  tutoria  versus  hypertext  instructional  systems.  Examination  of  post- 
test  performance  when  viewed  in  relation  to  treatments  and  ability  groupings  revealed 

SSSLfiXfiF  dH8re.r)Ces  in  P0^68' scorM  ware  ^ndSn^Vee 
tt.flSupif ln  tneJWertext  treatment.  Even  though  practice  using  a  similar 

.ffiSS^Sr^f  i£ne,prior,to  treatment-  tne  objects'  lack  of  experience  and 
unfamiiiarity  with  non-linear  learning  may  have  been  a  factor  In  these  results  Another 

SSESS^S? the  Ln$t!ument  used  ,or  evaluatlon-  P«rhap7ad^r?rneSf 

*f     emphasizes  the  broad  scope  of  potential  associations  made  In 
hypterext  environments  may  be  more  suitable.  Significant  results  were  obtained 

ETI522 ?i?W  a  -  hi?D  *B,y  °roups  38  we"  as  tne  avera0*  and  hlflb  «blH«y  groups  in 
the  tutoria  treatment.  These  results  indicated  the  tutorial  was  most  effective  fo?  high 
acuity  subjects.  * 

h0hw-~  ™!  8tudy        attempt  t0  assess  tne  e«ectlveness  of  tutorial  versus  hypertext 
delivery  systems  and  to  identify  learner  characteristic  which  may  be  used  to  prescribe  the 
most  beneficial  mode  of  instruction.  Overall,  findings  indicate  learner  control  was  not  a 
K£ ♦ne.nypstext  mode  of  instruction.  Additionally,  ability  appears  to  play  a  key  role 
mSi  SJT  °f  try°V  a$  ^^"nined  by  the  posMeSSn^S I S3 
hypertexTrTiode  P««formance  as  determined  by  the  post-test  instrument  in  the 

. « .^J8  8tudy  set  ,ortn  addittonal  questions  to  be  answered  through  research.  What 

%Z£ii£2Fl!?*  e<!ectK/ely  assesses  learnin° in       «*°™E£ ciT 

S22?Je.am.,nBltyS8  or  learoer  characteristics  be  used  to  predict  success  In  a  tutorial  or 
SHIS  Sy8tem?  o1^68  novetty  D,ay  a  ro,e  ln  ^"al,  hypertext,  or  both  forms  of 
.Sf1^1?7  „Ca»n  c»uantitatlva  arw'ysis  of  audit  trails  provide  insight  to  designers 
of  hypertext  systems?  Answers  to  these  questions  and  others  must  be  sought  to  validate 
the  use  of  hypertext  as  an  instructional  system. 
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7. 

Compsnaatlon  for 

Infrlngsmsnt 
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4. 

Eligibility  Rsqulremsnts 

a. 

Registration 

5. 

9. 

'"Author/Inventor 

What  Constitute* 

Chsllsngss*" 

Infringement? 

10. 

Quit 

Figure  1 .  Tutorial  Main  Menu 


Figure  2.  Hypertext  Main  Msnu 
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Introduction 

According  to  the  latest  version  of  the  Encyclopedia  of  Educational  Research 
(Allan,  1992),  there  are  currently  "three  mtgor  paradigms,  or  three  different  ways 
of  investigating  important  aspects  of  education"  (p.  620)  in  use  by  educational 
researchers  (Soltis,  1992).  These  paradigms  are  described  as: 

•  the  positivist  or  quantitative  paradigm, 

•  the  interpretivist  or  qualitative  paradigm,  and 

•  the  critical  theory  or  neomarxist  paradigm. 

Soltis  (1992)  concludes  that  these  paradigms  currently  coexist  within  the 
educational  research  community  "in  a  state  of  tolerance  and  struggle"  (p.  621). 
Some  view  the  three  paradigms  as  necessary  to  yield  a  full  and  complete  picture  of 
educational  phenomena  (cf.,  Salomon,  1991)  whereas  others  regard  them  as 
incompatible  (cf.,  Cziko,  1989, 1992),  or  as  Kuhn  (1970)  would  put  it, 
incommensurable. 

There  is  another  perspective,  espoused  by  Yeaman  (1990)  among  others,  that 
educational  research  (along  with  the  other  behavioral  sciences)  is  what  Kuhn 
called  pre-paradigmatic.  According  to  this  perspective,  educational  research  lacks 
the  components  of  a  Kuhnian  research  paradigm  such  as  strong  theory,  valid 
measurements,  unambiguous  research  designs,  and  reliable  analytical  processes. 

Scientific  research  that  is  conducted  without  the  structure  of  a  supporting  scientific 
paradigm  should  be  labeled  "pseudoscience"  in  that  such  research  is  false, 
deceptive,  or  sham  science.  The  title  of  this  paper  is  meant  to  put  forth  the  thesis  that 
some,  perhaps  most,  research  in  the  field  of  instructional  technology  is 
pseudoscience.  It  is  beyond  the  scope  of  this  paper  to  provide  a  comprehensive 
critique  of  the  complete  spectrum  of  research  in  the  field  of  instructional 
technology.  Instead,  this  paper  presents  a  critique  of  that  type  of  instructional 
technology  research  that  is  based  upon  what  Soltis  (1992)  called  the  positivist  or 
quantitative  paradigm.  More  specifically,  this  paper  focuses  on  a  critique  of  a  body 
ot  quantitative  inquiry  known  in  the  field  as  "leaner  control  research" 
(Steinberg,  1977, 1989). 
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Caveats 

Before  presenting  a  critique  of  learner  control  research,  a  few  caveats  are 
necessary.  First,  calling  instructional  technology  research  "pseudoscience"  may 
seem  overly  harsh  because  it  suggests  that  the  people  engaged  in  this  research  are 
corrupt.  The  following  critique  should  in  no  way  be  interpreted  as  suggesting  that 
the  authors  of  the  studies  reviewed  have  engaged  in  any  wrong-doing.1  Instead  I 
believe  that  they  have  followed  powerful,  albeit  misguided,  dictates  to  be  scientific 
in  the  same  way  that  older,  more  established  sciences,  are  scientific.  The  desire  to 
be  scientific  is  endemic  in  our  field  and  many  closely  related  educational  and 
psychological  fields.  This  desire  has  many  motivations,  not  the  least  of  which  is 
survival  in  a  "publish  or  perish"  academic  culture  (Yeaman,  1990). 

Second,  this  critique  is  not  about  the  debate  between  the  merits  of  basic  versus 
developmental  research.  A  more  serious  problem  in  our  field  is  that  so  much  of  the 
research  that  is  supposed  to  be  basic  subscribes  to  the  concepts,  procedures,  and 
analytical  schemes  of  positivist,  quantitative  inquiry  without  meeting  the 
theoretical  and  statistical  requirements  of  that  paradigm  (Orey,  Garrison,  & 
Burton,  1989).  In  short,  the  problem  is  really  one  between  valid  and  invalid 
research,  or  as  described  in  this  paper,  science  and  pseudoscience. 

Third,  this  critique,  while  not  wholly  original,  should  not  be  viewed  as  redundant, 
as  was  suggested  by  one  of  the  reviewers  of  this  paper.  Although  many  critiques 
have  been  made  of  positivist  inquiry  in  education  in  general  (cf.,  Criko,  1989, 1992; 
Phillips,  1987)  and  instructional  technology  in  particular  (cf.,  Clark,  1989;  Reeves, 
1986, 1989),  studies  of  this  type  continue  to  be  pervasive.  Indeed,  hardly  an  issue  of 
the  research  journals  in  our  field  (cf.,  Educational  Technology  Research  and 
Development,  Journal  of  Computer-Based  Instruction,  Journal  of  Research  on 
Computing  in  Education)  appears  without  one  or  more  reports  of  these  types  of 
studies.  ^ 

Learner  Control 

Learner  control  is  defined  as  the  design  features  of  computer-based  instruction3 
(CBI)  that  enable  learners  to  choose  freely  the  path,  rate,  content,  and  nature  of 


1  Of  course,  only  the  most  naive  person  would  maintain  that  there  is  no  corruption 
involved  in  instructional  technology  research  just  as  there  is  in  other  types  of 
scientific  and  pseudoscientific  inquiry.  It  is  not  unusual  to  find  out  that  scientists 
in  the  "hard"  sciences  as  well  as  medicine,  representing  highly  respected  research 
institutions,  have  faked  data  or  intentionally  misinterpreted  findings  (Sykes, 
1988).  This  no  doubt  occurs  in  our  own  field  as  well. 

2  This  is  hardly  surprising  given  that  how  people  conduct  research  "is  based 
largely  on  the  skills  and  ideas  that  are  tacitly  transmitted  during  what  could  be 
called  a  scientist's  apprenticeship,  in  graduate  school  for  example"  (Bohm  &  Peat, 
1987,  p.  52).  Many  members  of  the  tenured  generation  of  instructional  technology 
faculty  who  dominate  the  teaching  of  research  methods  in  instructional  technology 
graduate  programs  were  schooled  in  positivist  methods  and  may  be  unfamiliar 
with  or  uncomfortable  with  alternative  approaches. 

3  Computer-based  instruction  is  used  rather  loosely  in  this  paper  to  include  a  wide 
range  of  interactive  learning  systems,  ranging  from  a  simple  tutorial  to  a  complex 
learning  environment. 
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feedback  in  instruction.  Learner  control  is  contrasted  with  "program  control 
i.e.,  design  features  that  determine  the  path,  rate,  content,  and  feedback  in 
instruction  for  learners.  Learner  control  and  program  control  represent  a 
continuum  of  learner  latitude  that  varies  widely  both  within  and  between  different 
instances  of  CBI. 

Proponents  of  CBI  claim  that  CBI  allows  students  to  be  in  control  of  their  own 
learning,  and  further,  that  being  in  control  of  learning  results  in  greater 
achievement.  Although  this  popular  belief  is  often  debated,  the  research  evidence 
for  and  against  these  claims  is  extremely  weak  because,  as  documented  below, 
inappropriate  research  methods  have  been  used  to  investigate  issues  of  learner 
control.  A  m*jor  thesis  of  this  paper  is  that  most  of  the  learner  control  research 
currently  reported  constitutes  "pseudoscience"  in  that  it  fails  to  meet  the  m*jor 
theoretical  and  methodological  assumptions  underlying  accepted  research 
methodologies  within  the  positivist,  quantitative  paradigm  (Orey  et  al.,  1989). 

Given  the  inadequacy  of  the  research  conducted  to  date,  it  should  not  be  surprising 
that  research  support  for  learner  control  is  inconsistent.  Steinberg  (1977, 1989) 
sums  it  up  as  follows: 

Learner  control  sometimes  resulted  in  greater  task  engagement  and  better 
attitudes,  but  not  necessarily  in  greater  achievement.  In  some  instances 
learner  control  led  to  worse  performance  than  computer  control.  While  many 
students  were  motivated  by  learner  control,  others  were  indifferent  to  it. 
Aptitude  and  trait-treatment  research  yielded  no  definitive  conclusions  about 
learner  control  (Steinberg,  1989,  p.  117). 

Ross  and  Morrison  (1989)  conclude,  "research  findings  regarding  the  effects  of 
learner  control  as  an  adaptive  strategy  have  been  inconsistent,  but  more  frequently 
negative  than  positive."  Ross  and  Morrison  claimed  that  at  least  part  of  the 
problem  may  be  insufficient  precision  in  defining  just  what  is  meant  by  "learner 
control"  in  various  studies,  and  that  learner  control  probably  is  not  "a  unitary 
construct,  but  rather  a  collection  of  strategies  that  function  in  different  ways 
depending  on  what  is  being  controlled  by  whom." 

Problems  with  Definitions  of  Learner  Control 

Learner  control  is  a  concept  that  seems  to  mean  something  very  clear  and 
important,  but  it  is  so  loosely  defined  in  practice  as  to  mean  very  little.  Of  course, 
in  the  scientific  sense,  learner  control  is  a  construct.  A  construct  is  a  concept  that 
has  been  intentionally  synthesized  on  the  basis  of  observation  and/or  theory  to 
represent  an  idea  or  a  phenomenon.  As  a  scientific  construct,  we  must  be 
concerned  with  how  learner  control  is  defined  and  measured. 

Learner  control  has  been  one  of  the  most  heavily  researched  dimensions  of  CBI  in 
recent  years  (Steinberg,  1989).  Figure  1  illustrates  the  learner  control  dimension 
in  CBI  ranging  from  complete  program  control  to  unrestricted  learner  control. 


Learner  Control 


Non-existent  Learner  Control  Unrestricted  Learner  Control 

(Program  Control) 


Figure  1.  Learner  control  dimension  of  CBI. 
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Upon  first  consideration,  learner  control  may  seem  like  a  simple  concept ;  in  that  it 
refers  to  the  options  in  CBI  that  allow  learners  to  make  decisions  about  what 
sections  to  study  and/or  what  paths  to  follow  through  interactive  material. 
However,  more  careful  consideration  indicates  that  learner  wntrol I.,  awry 
complex  issue.  For  example,  whenever  learner  control  is  denned,  we  must  ask 
Survive,  what  is  the  learner  controlling.  In  some  cases,  it  may  simply  be  the  ^te 
or  order  of  screen  presentations.  In  other  cases,  it  may  involve  the  activation  of 
multiple  microworlds  and  support  systems  within  a  complex  learning 
environment  in  which  the  learner  virtually  authors  his/her  own  CBI. 

The  "control  of  what"  question  is  a  critical  one  if  we  ever  wish  to  have  any 
meanSu?  impart  on'the  design  of  CBI  (Ross  &  Morrison,  1989).  The  finding  that 
learner  control  of  context  for  examples  is  effective  in  one  setting  and  another 
finding  that  learner  control  of  amount  of  review  is  effective  in  another  setting 
prTdes  lUtle  basis  for  linking  these  two  types  of  learner  control  into mj. meaningful 
prescription  for  design  of  CBI.  Even  less  guidance  is  provided  tar  link 
dimension  of  learner  control  with  other  critical  dimensions  of  CBI,  e.g.,  structure. 

Hannafin  (1984)  needed  eight  qualifying  statement*  to  W^*^*™^^ 
learner  control,  and  the  recent  research  of  Ross  and  Morrison  (1989)  and  others 
indicates  that  even  more  specific  qualifiers  may  be  needed  to  clarify  the 
effectiveness  of  learner  control  in  different  contexts.  Can  such  complex  statements 
be  logically  integrated  with  equally  complex  explications  of  other  relevant 
variables  to  provide  a  clear  basis  for  interactive  design  decisions?  When  do  such 
Zplexfties  begin  to  exceed  the  human  conceptual  capacity  to  deal  with  sever,  plus 
or  minus  two  variables  at  one  time?  Or  are  we  in  the  situation  that  Cronbach  (1975) 
so  eloquently  described  of  not  being  able  to  pile  up  generalizations  fast  enough  to 
keep  up  with  the  process  of  learning? 

It  seems  that  researchers  in  instructional  technology  have  come  to  use  terminology 
such  as  learner  control  without  sufficiently  examining  the  meaning  and 
implications  of  the  construct.  It  is  almost  as  if  the  mere  utterance 
learner  control  brings  it  into  meaningful  existence.  However  the  philosopher 
Alfred  ItoSybsW  Wted  in  Bohm  and  Peat,  1987)  describes  the  danger  of  this  way  of 
thinking.  Korzybski  noted  that  our  observations  of  the  world  around  U8  are i  limited 
both  by  the  frailties  of  our  perceptual  senses  and  our  existing  knowledge  tha  sha jes 
our  interpretation  of  sense  data.  Further,  he  maintained  that  every  land  o  thought 
conceived  as  a  result  of  observations  is  an  abstraction  that  reveals  only  |  "rnited 
view  or  slice  of  reality  as  we  perceive  it.  Words  or  concepts  in  turn  are  abstractions 
of  thought,  and  as  such,  fail  to  capture  the  complete  essence  of  a  thought 
Consciously  constructed  concepts  such  as  "learner  control  are  even  ^to- 
removed  from  our  sense  data,  thoughts,  and  concepts.  °™™*™"°[™™fC 
constructs  represent  yet  another  layer  of  abstraction  and  the  mathemat  icsand 
Statistics  with  which  we  analyze  the  data  collected  with  our  measures 
additional  abstraction.  Finally,  even  more  abstraction  »  : "W**^**™ 
through  the  chain  of  abstractions  to  relate  statistics  to  conclusions  about  the  nature 
of  reality. 

All  this,  of  course,  is  the  very  "stuff'  of  science.  Well-established  defences  such  as 
physics  deal  with  these  very  same  problems.  The  solution  of  these  Problemsin 
physics  required  the  development  of  a  series  of  scientific  paradigm" ™  replaang 
the  other  with  a  more  or  less  cohesive  infrastructure  of  tacit  beliefs  and  methods. 
The  construction  of  paradigms  in  physics  did  not  occur  overnight,  and  some 
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rfll!!?       1  6y      faufr0m  mUhA  (cf-  ^P"-  1987>  For  "ample,  the  concept 
ofatoms was  .round  for  thousand,  of  year,  before  it  wu  considered  Seriously. 

o^limn^Tn0"^  TdeT^inS  °f  ftt°m'  "  iiiht]y  compacted  concoctions 
of  '"d  neUt?n'  b"**1*?*  t°  •  concept  of  atoms  containing  vast  volumes 

1988)  8P      *  P0pu,ated  by  ""batomic  particlea  of  many  kinds  (Pagels, 

il?^  above.      is  not  the  first  paper  to  suggest  that  instructional  technology 
research  and  educational  research  in  general  suffer  from  a  lack  of  well- 
established  paradigms  (cf.,  Yeaman,  1990).  However,  being  "pre-paradigmatic " 

kZ  SSSS «Z UnH:??b,e  8taute-  ^  are  at  ,ea8t  tw°  ^tivtsTfhis. 
2S\2f W  ma,nta,"ed  that  even  the  most  established  sciences  such  as  physics 
«£n  ^  ^  P™-pa™d,fn  sta«f'  and  that  most  contemporary  social  sciences  are 

51?     ?  T  Kuh"  w°ndered  why  social  8cience  researchers  worry  so  much 
about  whether  they  are  really  scientists: 

Inevitably  one  suspects  that  the  issue  is  more  fundamental. 

Probably  questions  like  the  following  are  really  being  asked:  Why 

does  my  field  fail  to  move  ahead  in  the  way  that,  say,  physics  does? 

What  changes  in  technique  or  method  or  ideology  would  enable  it  to 

do  so?  These  are  not,  however,  questions  that  could  respond  to  an 

agreement  on  definition.  Furthermore,  if  precedent  from  the 

natural  sciences  serves,  they  will  cease  to  be  a  source  of  concern  not 

when  a  definition  is  found,  but  when  the  groups  that  now  doubt  their 

own  status  achieve  consensus  about  their  past  and  present 

accomplishments,  (pp.  160-161) 

Kuhn's  emphasis  on  reaching  consensus  within  a  paradigm  is  intriguing.  As 

re^h?™nfementt--  ?fleCting  thC  ,Brger  debate  O"1*™*  among  educational 
r    7ry  k,"d.  some  instructional  technologists  view  the  past  several 
decades  of  media  research"  as  having  extremely  limited  value  (cf.,  Clark  1989) 

WnwT  g  ?  foundat,ons  for  Present  day  inquiry  (cf.,  Ross  &  Morrison,  1989). 
to  "Hf  """f *  cement  concerning  these  issues  before  we  can 

n!«mL,  J , °'  I  W°U,d  h0pe  that  we  remain  in  the  controversial  pre- 
paradigmatic  stage  for  some  time  to  come. 

Another  perspective  on  the  pre-paradigmatic  stage  is  provided  by  two  ohvsicists 

oaarsdi^,and     °aVidHPeat  (1987)"  They  ma*ntain  that  over-emphas  on 
W ft      PU"^  W  10  Wame  for  much  of  the  fragmentation  in  physics  and  other 

to  the  cTatiZ         I'1?""  T  8  Paradigm  Can  ^  and  °ften  *8'  detrimenta 
mi lte?D?Syn  ^rheS fauthentic  ^nr.  They  claim  that  Kuhn  is  often 
Kuhn^S      wPPorhw  of  paradigm  changes.  Bohm  and  Peat  argue  that 

«3E?  Paradi£mS  n0t  88  fundamental  theories,  but  as^a  whole  way 
^  »1/d      k,n£,JCOummun,Cat,ngl  and  Perceiving  with  the  mind"  (p.  52). 
ZZSE?       TTTd  the  tecit  or  unc°n«iou8  consent  that  pervades  a  given 

thl  tZZ       J  *?°  h/?Vy  a  price  on  the  mind  and  the  "hutting  down 

the  imagination  and  playfulness  that  the  mind  requires  for  meaningful  inquiry 

f  ourcS^f^H  8  S?entifiC  Paradigm  means  that  we  ™*  aSfice's  me 
We  ^L7^ S'i  T    P,r,eff  10  remain  Pre-paradigmatic  for  a  long  time  to  come. 

EducTtionT^  B?  aC  B"  e,8e'  ^  R°bert  Ebe1'  a  past  Pre8ident  of  the  AmerfcTn 
tSZ  J-   ?  Research  Association  (cited  in  Parley,  1982)  pointed  out  "  it 
^ducation)  is  not  in  need  of  research  to  find  out  how  it  works.  It  is  in  need  of 
creative  invention  to  make  it  work  better"  (p  11) 
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Returning  to  the  difficulty  of  denning  learner  control,  at  least  part  of  the  problem 
stems  from  the  lack  of  an  infrastructure  of  tacit  understandings  and  methods  to 
support  this  construct.  In  other  words,  we  lack  a  scientific  paradigm .to  guide 
research  regarding  learner  control.  However,  as  described  below,  this  has  done 
little  to  dampen  the  ardor  of  researchers  in  our  field. 

Problems  with  Learner  Control  Research 

As  indicated  in  the  reviews  by  Ross  and  Morrison  (1989)  and  Steinberg  (1989), 
increasingly  large  numbers  of  research  studies  have  been  earned  out  to 
investigate  the  relative  effectiveness  of  learner  control.  These  investigations  are 
focused  on  isolating  the  attribute  or  dimension  of  CBI  called  learner  control  and 
estimating  its  effectiveness  in  a  variety  of  implementations  (e.g.,  learner  control 
with  advisement  versus  learner  control  without  advisement.)  However  learner 
control  studies  have  often  led  to  no  significant  results  in  terms  of  the J5»*cted 
main  effects  (cf.,  Kinzie  &  Sullivan,  1989;  Lopez  k  Harper,  1989;  McGrath,  1992; 
Ross,  Morrison,  &  O'Dell,  1989;  Santiago,  1990).  In  fact,  technical  and 
methodological  flaws  largely  invalidate  these  studies.4 

First  it  is  clear  that  the  instructional  treatments  used  in  these  studies  are  usually 
far  too  brief  to  provide  learners  with  sufficient  experience  for  learner  control 
variables  to  be  "actualized."  Kinzie  and  Sullivan  (1989)  reported  that  mean 
completion  times  for  the  experimental  and  control  CAI  lessons  were  29  minutes, 
four  seconds  for  the  former  and  29  minutes,  six  seconds  for  the  latter,  and  Lopez 
and  Harper  (1989)  estimated  that  their  CBI  lesson  treatment  was  approximately  30 
minutes  in  length."  The  treatments  experienced  by  the  students  in  the  Ross. 
Morrison,  and  O'Dell  study  (1989)  averaged  between  25  and  30  minutes,  and  the 
students  in  the  Santiago  (1990)  study  averaged  30  minutes  in  the  actual  CBI  lesson 
after  a  15-20  minute  orientation  to  the  computer  presented  by  the  researcher. 
McGrath's  (1992)  students  averaged  only  13  to  17  minutes  in  their  hypertext, 
computer-assisted  instruction,  no-choice  CAI,  or  paper  treatments. 

Cronbach  and  Snow  (1977)  cautioned  that  ten  or  more  separate  interactive  sessions 
were  necessary  to  acquaint  students  with  innovative  instructional  treatments. 
How  can  a  dimension  as  complex  as  learner  control  be  expected  to  have  an  effect  in 
one  session  treatments  lasting  less  than  an  hour?  Suppose  that  a  medical 
researcher  was  investigating  the  effects  of  an  aspirin  regimen  on  subsequent  heart 
disease.  The  medical  researcher  might  be  interested  in  varying  repmens  of 
aspirin  (one  a  day  versus  one  every  other  day),  but  he/she  would  not  look  for  the 
effects  of  administering  a  single  aspirin  on  one  occasion.  And  yet,  we  seem  to  be 
willing  to  place  students  in  an  analogous  situation  vis  a  vis  learner  control  and 
expect  meaningful  results. 

Second  there  is  a  lack  of  consequential  or  relevant  outcome  measures  in  many 
media  replication  studies.  Kinzie  and  Sullivan  employed  CAI  lessons  on  solar 
energy  and  tarantulas,  subjects  not  clearly  articulated  with  their  regular  science 
curriculum.  Although  the  seventh  and  eighth  grade  students  in  the  Lopez  and 
Harper  (1989)  study  were  told  that  their  grades  would  be  reported  to  their  teacher,  the 
thirty  minute  lesson  on  insects  was  unrelated  to  their  normal  curriculum.  The 


<  The  five  studies  selected  for  this  critique  are  representative  of  a  larger  collection 
of  similarly  flawed  studies.  A  comprehensive  analysis  of  available  learner 
control  studies  is  underway. 
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)imenfilJa«oU«avte  t!acher  education  8t"dent  -volunteers"  in  the  Ross,  Morrison,  and 
ODell  (1989)  and  the  Santiago  (1990)  studies  received  "credit"  toward  course  grades 
for  participating  in  the  research,  but  the  content  subjects  were  only  indirectly 
related  to  the  courses  in  which  the  students  were  enrolled.  McGrath's  (1992) 
subjects  were  also  undergraduate  teacher  preparation  students  enrolled  in  a  media 
course,  but  the  subject  of  the  treatment  lesson  involved  the  mathematics  of 
determining  surface  areas  of  hollow  figures. 

Subjects  in  learner  control  research  should  be  engaged  in  learning  that  is 
personally  meaningful  and  that  has  real  consequences  for  them.  Further, 
volunteer  subjects  learning  content  unrelated  to  their  education  or  training  needs 
or  interests  are  inappropriate  because  this  practice  introduces  unacceptable  threats 
to  internal  and  external  validity  (Isaac  &  Michael,  1971).  It  is  ironic  that  so  many 
learner  control  studies  utilize  researcher-created  interactive  materials  when  a 
number  of  large  scale  interactive  learning  systems  are  available.  Examples 
include  Macintosh  Fundamentals,  an  interactive  videodisc  training  course 
developed  by  Apple  Computer,  Inc.  (1990)  and  Illuminated  Books  and  Manuscripts, 
a  set  of  five  interrelated  educational  multimedia  programs  produced  by  the  IBM 
Corporation  (1991).  Finding  situations  in  which  these  and  similar  large  scale 
programs  are  used  in  realistic  contexts  may  be  challenging  for  researchers,  but  the 
alternative  compromises  exact  too  great  a  price. 

Two  other  concerns  are  small  sample  sizes  and  large  attrition  rates  that  leave 
some  cells  in  the  analyses  with  inadequate  numbers  of  subjects.  Another  problem 
concerns  dropping  subjects  from  the  analyses  who  have  correctly  answered  all 
questions  within  the  interactive  lessons,  as  was  done  in  the  Kinzie  and  Sullivan 
(1989)  research.  Without  missing  a  correct  response,  these  students  clearly  had  no 
opportunity  to  experience  learner  control  or  program  control  over  review.  And 
what  about  students  who  only  missed  one  or  two  of  the  fifteen  questions  interspersed 
into  the  CAI  lessons?  Did  they  "experience"  the  treatment  variables? 

Clark  (1989)  is  correct  in  suggesting  that  many  of  these  and  similar  studies  in 
instructional  technology  are  conceptualized  without  an  adequate  understanding  of 
all  the  related  theoretical  assumptions  and  pertinent  research  findings.  As  a 
result,  little  guidance  is  provided  for  linking  the  dimension  of  learner  control  with 
other  critical  dimensions  of  CBI,  e.g.,  screen  design  or  feedback.  The  conclusions 
presented  in  some  of  the  studies  cited  above  are  nothing  less  than  convoluted  The 
irony  is  that  in  reading  any  single  one  of  these  complex  "explanations"  of  why 
significant  findings  were  not  found,  the  reader  may  find  the  researcher  s 
arguments  somewhat  plausible.  The  real  complexity  arises  when  one  tries  to  make 
sense  of  these  explanations  across  the  various  research  reports.  In  the  end,  we  must 
ask  ourselves  how  such  complex  statements  can  be  logically  integrated  with 
equally  complex  explications  of  other  relevant  variables  to  provide  a  clear  basis  for 
CBI  design  decisions  (Reeves,  1992). 

What  should  be  done? 

This  paper  presents  evidence  that  learner  control  research  studies  are  flawed  in 
terms  of  sample  sizes,  treatment  duration,  content  selection,  and  other  theoretical 
and  methodological  issues  to  such  an  extent  that  the  research  has  little  value.  (The 
criticisms  of  learner  control  research  described  above  could  easily  be  extended  to 
other  areas  of  research  in  instructional  technology,  e.g.,  studies  on  screen  design 
issues.)  These  flaws  and  fundamental  problems  arising  from  a  lack  of  a 
scientific  paradigm  to  guide  research  in  this  field  yield  a  conclusion  that  the 
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positivist  research  we  are  currently  conducting  is  largely  pseudoscience.3 

However,  it  is  not  enough  to  critique  research  in  our  field.  I  wish  to  suggest  some 
new  directions.  First,  there  is  a  serious  need  for  graduate  students  and  researchers 
in  our  field  to  develop  an  improved  understanding  of  contemporary  philosophy  of 
science.  Courses  in  the  philosophy  of  science,  preferably  taken  outside  Colleges  of 
Education,  would  expose  researchers  to  a  larger  spectrum  of  approaches  to  inquiry 
than  found  in  many  traditional  courses  emphasizing  experimental  and  quasi- 
experimental  designs.  Phillips  (1987)  speaks  to  this  need,  "New  approaches  to  the 
design  of  evaluations  of  educational  and  social  programs  are  being  formulated 
that  make  the  "true  experiment"  seem  like  a  lumbering  dinosaur,  yet  some  folk 
persist  in  thinking  that  dinosaurs  are  wonderful  creatures"  (p.  viii). 

Second,  I  believe  instead  of  worrying  about  whether  or  not  we  have  a  paradigm  in 
which  to  conduct  research  in  instructional  technology,  we  should  begin  to  address 
some  of  the  questions  that  Kuhn  (1970)  suggests  underlie  our  concerns.  Important 
questions  include:  Why  hasn't  our  field  moved  ahead  as  others  have?  What 
techniques,  methods,  or  ideologies  will  enable  us  to  progress?  In  other  papers 
(Reeves,  1986, 1990, 1992),  I  have  suggested  that  we  explore  alternative  approaches  to 
research  in  our  field  such  as  computer  modeling  (Pagels,  1988)  and  formative 
experiments  (Newman,  1990).  These  alternative  approaches  address  systemic 
rather  than  analytic  issues  in  educational  contexts.  (Salomon  (1991)  provides  an 
excellent  overview  of  the  differences  between  systemic  and  analytic  approaches.) 
The  value  of  systemic  approaches  are  that  they  provide  the  basis  for  theory  and 
hypotheses  that  may  be  investigated  using  analytic  procedures.  Salomon  (1991) 
summed  the  issue  up  this  way,  "Without  observations  of  the  whole  system  of 
interrelated  events,  hypotheses  to  be  tested  could  easily  pertain  to  the  educationally 
least  significant  and  pertinent  aspects,  a  not  too  infrequent  occurrence"  (p.  17). 
Such  is  the  case  in  learner  control  research. 

Third,  the  time  to  explore  alternative  research  paradigms  in  our  field  has  never 
been  more  critical.  Instructional  technology  as  a  field  may  be  in  danger  of 
becoming  irrelevant.  One  of  the  field's  major  components,  instructional  design 
(Briggs,  Gustafson,  &  Tillman,  1991),  is  under  attack  in  several  quarters  (cf.t 
Carroll,  1990;  Gery,  1991).  Further,  some  of  the  most  advanced  learning 
environments  are  being  created  by  cognitive  psychologists  and  others  in  fields 
outside  instructional  technology,  such  as  the  Jasper  Series  at  Vanderbilt  University 
(Cognition  and  Technology  Group  at  Vanderbilt.  1992). 

Exploring  alternative  research  paradigms  might  also  encourage  us  to  undertake 
more  of  the  "creative  invention"  that  Ebel  claimed  is  needed  in  education  (cited  in 
Farley,  1982).  This  may  require  changing  the  reward  structures  in  traditional 
academic  settings.  It  is  pathetic  that  the  creation  of  an  interactive  learning  system 
with  the  potential  to  change  the  lives  of  hundreds,  even  thousands  of  people,  is  given 
less  worth  in  some  academic  settings  that  a  "research"  paper  published  in  the  most 


5  Some,  perhaps  many,  will  disagree  with  this  conclusion.  One  rebuttal  of  the 
thesis  that  learner  control  research  is  pseudoscience  might  involve  making  a  case 
that  the  example  studies  cited  are  simply  poor  studies,  and  that  studies  that 
eliminate  the  flaws  in  these  studies  by  incorporating  substantive  treatments  over  a 
meaningful  period  of  time  with  large  samples  in  practical  contexts  would  be  "real" 
science.  However,  such  a  defense  does  not  address  the  problems  of  defining 
learner  control  as  described  above. 
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As  noted  earlier  in  this  paper,  Soltia  (1992),  writing  in  the  Encyclopedia  of 
Educational  Research  (Alkin,  1992),  describee  three  major  paradigms  for  inquiry 
in  education,  the  positivist  (quantitative),  the  interpretivist  (qualitative),  and  the 
critical  theorist  (neomarxist).  I  don't  know  enough  about  the  neomarxist  paradigm 
to  comment  on  its  applicability  in  our  field.  However,  I  do  believe  that  we  could 
benefit  greatly  from  well-conceived  qualitative  inquiry  in  our  field.  I  recommend 
that  we  cal  a  moratorium  on  the  types  of  quantitative  studies  described  in  this 
paper,  replacing  them  with  extensive,  in-depth  efibrta  to  observe  human  behavior  in 
our  field  and  develop  meaningful  theory  that  may  later  be  susceptible  to 
quantitative  inquiry.  I  am  not  suggesting  that  we  eliminate  quantitative,  positivist 
inquiry  altogether.  After  all,  as  Salomon  (1991)  describes,  quantitative  (analytic) 
and  qualitative  (systemic)  inquiry  may  have  complementary  functions.  However, 
the  qualitative,  interpretivist  paradigm  should  proceed  the  quantitative  if  we  are  to 
identify  meaningful  hypotheses  to  investigate  empirically. 

The  difficulty  of  «mrse,  is  assuring  that  the  interpretivist,  qualitative  inquiry  is 

S^rT  .and,ng°r0U8ly  applied  There  ia  a  dan*er  that  researchers  in  our 
field  will  adopt  qualitative  methodologies  to  inquiry  as  poorly  as  they  have 
quantitative  methods.  No  one  becomes  an  ethnographer  or  participant  observer 
overnight  (Preissle,  1991).  Case  studies  and  ethnography  can  be  as  poorly 
conceived  and  conducted  as  any  other  methods  (Levine,  1992).  It  has  been  observed 
that  some  physics  graduate  students  and  young  physicists  gravitate  toward 
research  in  cutting  edge"  topics  such  as  subatomic  phenomenon  and  tabletop 
fusion  not  so  much  because  these  are  the  most  important  topics,  but  because  it  is  so 
much  easier  to  get  to  the  frontiers  of  these  areas  of  physics  than  it  is  to  develop  the 
background  m  older  areas  of  inquiry  such  as  solid-state  physics.  It  would  be  a 
shame  if  researchers  in  our  field  jumped  onto  the  qualitative  bandwagon  without 
adequately  preparing  themselves  for  the  journey. 

What  would  be  the  value  of  expanding  observation  and  reflection  in  our  field? 

Consider  the  following  quote  from  the  expedition  log  of  John  Steinbeck  (1941),  the 

noted  American  author  who  was  also  a  marine  biologist: 

We  knew  that  what  we  would  see  and  record  and  construct  would  be 
warped,  as  all  knowledge  patterns  are  warped,  first  by  the  collective 
pressure  and  stream  of  our  time  and  race,  second  by  the  thrust  of  our 
individual  personalities.  But  knowing  this,  we  might  not  fall  into 
too  many  holes  -  -  we  might  maintain  some  balance  between  our 
warp  and  the  separate  things,  the  external  reality.  The  oneness  of 

r pTwefnP^rr  to  Publi!h  0r  P?™hJ8  especially  powerful  for  doctoral  students  and 
new  faculty  members.  In  light  of  this,  one  reviewer  of  this  paper  suggested  that 
criticisms  aimed  at  research  conducted  by  doctoral  students  and  new  faculty 

£™t h°nf w*"  '^"I  "I  P™8entJed  in  this  paper.  However,  I  think  that  holding  the 
research  of  graduate  students  and  new  professors  to  a  lower  set  of  standards  would 

a  ',    u  '  1 18 un8Ul]ing  10  their  oflen  ™««ri«ble  capabilities.  Second, 
*    and  facu,ty  ™mberHd°  not  conduct  re8earch  in  a  vacu™.  carted  by  the 
support  and  review  of  their  colleagues.  This  is  especially  true  for  doctoral 
candidates  who  have  the  combined  research  expertise  of  an  entire  faculty 
committee  to  guide  their  research  efforts.  Third,  the  research  conducted  by  students 
and  new  professors  constitutes  a  very  large  portion  of  the  research  carried  out  in 
our  field  and  must  be  rigorously  assessed  if  the  field  is  to  advance 
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these  two  might  take  its  contribution  from  both.  For  example:  the 
Mexican  sierra  has  XVII-15-IX'  spines  in  the  dorsal  fin.  These 
can  easily  be  counted.  But  if  the  sierra  strikes  hard  on  the  line  so 
that  our  hands  are  burned,  if  the  fish  sounds  and  nearly  escapes  and 
finally  comes  in  over  the  rail,  his  colors  pulsing  and  his  tail 
beating  the  air,  a  whole  new  relational  externality  has  come  into 
being  -  -  an  entity  which  is  more  than  the  sum  of  the  fish  plus  the 
fisherman.  The  only  way  to  count  the  spines  of  the  Bierra  unaffected 
by  this  second  relational  reality  is  to  sit  in  a  laboratory,  open  an  evil 
smelling  jar,  remove  a  stiff  colorless  fish  from  formalin  solution, 
count  the  spines,  and  write  the  truth  T>.  XVII-15-IX."  There  you 
have  recorded  a  reality  which  cannot  be  assailed  -  •  probably  the 
least  important  reality  concerning  either  the  fish  or  yourself.  It  is 
good  to  know  what  you  are  doing.  The  man  with  his  pickled  fish  has 
set  down  one  truth  and  has  recorded  ;<n  his  experience  many  lies. 
The  fish  is  not  that  color,  that  texture,  that  dead,  nor  does  he  smell 
that  way.  (p.  2) 

Do  we  not  now  have  too  much  in  common  with  the  technician  in  the  lab 
enumerating  the  spines  of  a  dead  fish  when  we  carry  out  the  type  of  quantitative 
studies  described  above?  Further,  do  we  have  too  little  in  common  with  the 
fisherman  struggling  with  the  rocking  of  the  boat,  the  whipping  of  the  wind,  and  the 
thrashing  of  the  fish  on  the  line?  Fifty  years  of  quantitative  inquiry  in 
instructional  technology  has  provided  us  with  precious  little  to  guide  our  efforts  to 
enhance  education  and  training.  Let  us  try  some  other  ways. 

Author's  Notes 

I  thank  my  colleagues  and  graduate  students  at  The  University  of  Georgia  for  their 
reviews  of  this  paper,  especially  Dr.  Mike  Hale,  Dr.  Mike  Orey,  and  Tengku 
Shahdan. 
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Introduction 

Emerging  as  a  separate  field  in  the  1940's,  Educational  Technology  drew  upon  theory  bases 
from  psychology,  learning,  and  perception.  The  field  was  derived  primarily  from  behavioral 
and  cognitive  psychology,  and  consequently  based  its  seminal  research  on  strict  experimental 
models  appropriate  to  the  early  questions  and  hypotheses  developed.  Media  was  tested 
experimentally,  and  found  (at  the  time)  to  be  effective. 

In  the  forty  years  of  research  that  followed,  the  same  experimental  paradigm  predominated. 
New  media  were  tested  against  old,  characteristics  of  learners  and  specific  media  were 
compared,  and  relationships  between  learners  and  media  were  explored,  all  using  various 
accepted  experimental  designs. 

More  recently,  the  field  has  broadened  its  definition  to  include  instructional  design,  media 
analysis,  and  learner  attitudes  among  other  topics.  Researchers  are  asking  a  variety  of  new 
questions,  many  of  which  would  be  difficult  to  examine  using  traditional  experimental 
methods. 


Background  and  Rationale 

Currently,  most  of  the  published  scholarship  in  educational  technology  has  been  based  upon 
experimental  and  descriptive  studies.  The  leading  researchers,  those  who  train  future  scholars 
m  research  methods,  have  only  recently  begun  to  accept  a  full  range  of  research  methodologies 
for  educational  technology.  Consequently,  educational  technology  journals  publish  few  studies 
based  upon  non-experimental  designs.  Since  reports  of  alternate  methodologies  are  few 
researchers  in  educational  technology  have  only  the  models  of  research  reports  in  other  fields 
to  assist  them  in  research  design  and  reporting. 

The  experimental  "bias"  in  educational  technology  has  been  questioned  by  many  researchers 
seeking  to  expand  the  areas  of  scholarship  in  the  field.  Becker  (1977)  recommended  alternate 
methodologies  to  approaching  educational  technology  research.  Cochrane,  et  al  (1980) 
suggested  that  researchers  base  new  areas  of  inquiry  on  "the  ethnography  of  situations  in  which 
people  use  visual  materials  (an  anthropological  approach)"  (p.  247).  They  stressed  the 
importance  of  recognizing  that  visual  learning  is  a  cultural  phenomenon  anJ  should  be  studied 
with  techniques  and  analyses  appropriate  to  cultural  processes.  Heinich  (1984)  in  his  N  I  E 
funded  ten  year  review  paper  encouraged  researchers  to  engage  in  more  "naturalistic"  inquiry. 

IHrough  the  use  of  naturalistic  inquiry,  I  am  sure  we  will  discover  important  factors  .  that 
have  been  ignored  too  long  ..."  (p.  84).  Heinich  also  argued  that  such  research  should  be 
encouraged  in  dissertation  work  and  should  be  more  disciplined  and  more  perceptive  than 
experimental  studies. 

Alternative  methodologies  would  lead  the  field  of  educational  technology  to  new  questions,  and 
to  often  ignored  areas  such  as  the  impact  of  educational  technologies  on  social  relationships 
and  educational  institutions.  Kerr  (1985)  suggested  that  methods  drawn  from  sociology,  policy 
sciences  and  anthropology  could  "shed  new  light  on  problems  that  have  traditionally  been 
approached  using  psychological  research  methods"  (p.  4).  Kerr  felt  that  asking  new  questions 
in  less  traditional  ways  was  critical  to  the  future  of  education. 
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This  growing  need  for  studies  that  do  not  appear  to  fit  the  tradition^  exrxmn^Ul  paradigm 
w  hSTToHiized  Yet  training  of  researchers  continues  in  the  traditional  vein. 
StZSgSL  e^tSTkLwledge  and  experience  primarily  in  expenmental  research 
studies  (of  the  tmlitional,  single  variable,  hypothesis  testing  variety). 

that  supported  this  goal  were  as  follows: 

To  acquaint  students  with  the  realm  of  noncausal  naturalistic  inquiry  and  its 
rel.S  to  hypotLesis  testing  research,  in  the  context  of  philosophy  of 
science; 

To  enable  students  to  raise  noncausal  issues  in  development  and  evaluation 
and  discuss  ways  of  investigating  these  issues  via  nontradmonal  paradigms, 

To  enable  students  to  analyze  ongoing  research  in  education  that  does  not 
conform  to  the  traditional  paradigms; 

4  To  give  students  practice  in  conceptualizing  research  questions  thatdo  not 
conform  to  the  traditional  paouiigms  and  in  designing  appropriate  studies  for 
investigating  these  questions;  and 

5  To  provide  a  forum  for  proposing  and  discussing  additional  and  alternative 
types  of  research  that  may  appear  warranted  by  growing  research  needs  in 
education. 

Workshop  topics  included: 

Background  to  Qualitative  Research-Why  and  How  it  has  been  taught 
Types  of  Qualitative  Research  (among  nmy)-Definition^xamples 
Case  Study 
Ethnography 

Grounded  Theory/Phenomenology 
Narrative  Inquiry 
Others  (Semiotic,  Critical  Inquiry) 
Topics  for  Educational  Technology  research  -  Selecting  appropriate  method 
Data  Gathering  Techniques 
Interview 
Observation 
Participant-Observation 
Document  Review,  Literature  analysis 

-Practice  data  gathering  activity  and  discussion 
Problems  and  Concerns 
Analysis 

Writing  and  reporting 
Validity  and  Reliability 
Absence  of  "Rules*  for  reporting 
Questions  and  Discussion 
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QUALITATIVE  RESEARCH:  DEFINITIONS 


Case  Study:  a  research  strategy  used  "when  'how'  or  why'  questions  are  being 
y  posed,  when  the  investigator  has  little  control  over  events,  and 
when  the  focus  is  on  a  contemporary  phenomenon  within  some 
real-life  context"  (Yin,  1989,  p.  13). 

Ethnography:  "the  art  and  science  of  describing  a  group  or  culture"  (Fetterman, 
1989,  p.  11). 

Holistic  ethnography:  used  "to  describe  and  analyze  all  or  part  of  a  culture 
or  coLLrdty  by  describing  the  beliefs  and  practices  o f  th, .  group 
studied  and  showing  how  the  various  pa rts  contrib ute  to  the 
culture  as  a  unified,  consistent  whole  (Jacob,  1987,  p.  10). 

Ethnography  of  communication:  also  referred  to  as  "microethnography"  and 
8    '•constitutive  ethnography,"  this  strategy  focuses  "on  the  patterns  of 
social  interaction  among  members  °f  a  cultural  group  or  among 
members  of  different  cultural  groups  (Jacob,  1987,  p.  18). 

Cognitive  anthropology:  also  called  "ethnoscience"  and  ^^¥3^'" 
this  method  is  used  to  study  culture  in  mentahstic  terms.  The 
approach  seeks  to  understand  participants'  cultural  categories  and 
Identify ^organizing  principles  that  underlie  these  categories 
(Jacob,  1987,  p.  22). 

Grounded  Theory:  the  development  of  theory  "from  the  bottom  up  (raather 
than  from  the  top  down),  from  many  disparate  pieces  of  collected 
evidence  that  are  interconnected"  (Bogdan  and  Biklen,  1992,  PP31- 
32). 

Phenomenology:  a  process  of  inquiry  used  to  "understand  the  meaning  of 
events  and  interactions  to  ordinary  people  in  particular 
situations....Phenomenologists  believe  that  multiple  ways  of 
interpreting  experiences  are  available  to  each  of  us  through 
interacting  witfc  others,  and  that  it  is  the  meaning  of  our 
experiences  that  constitutes  reality"  (Bogdan  and  Biklen,  1992,  p. 
34). 

Critical  Inquiry:  a  mode  of  inquiry  that  "focuses  on  criticism  in  the  sense  of  art 
^  9  and  literary  criticism  models  within  the  humanities"  (Hlynka  and 

Belland,  1991,  p.  6). 
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iu  worK  just  as  hard  on  determining  how  to  understand  thoc* 
und        reiati?n  t0  ^  VaSt  and  comP^x  netwo^ of  code  that 

educational  and  ,arger  cu,ture 

Policy  Research:  "the  process  of  conducting  research  on,  or  analysis  of  a 

fundamental  social  problem  in  order  to  prortde  poHctmakers  with 

Participatory  Action  Research:  a  research  strategy  in  which  "some  of  the 

people  ui  the  organization  or  community  under  studTparHcioate 

process  from  the  initial  design  to  the  final  presentation  of  results 
and  discussion  of  their  action  implications"  (Foote  Whyte 
Greenwood,  and  Lazes,  1991,  p.20).  ™nyie, 

2S£T t,Wt  th6Se  am0aCkeS  '°  I"*"*'™  "  W  are  not  all  Mutually 
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A.E.C.T.  1993 

Robinson/Driscoll 

Qualitative  Research  Workshop 

Readings  on  Qualitative  Research 
An  Annotated  Bibliography 

Phyllis  Baker 
Northern  Illinois  University 

All  of  the  books  listed  below  involve  discussions  of  qualitative  research  in  general  and 
eth.ograph  b  pair.  The  most  beneficial  aspect  of  each  book  is  highlighted.  Ultimately  th.s 
iSl  be  utilized  to  describe  and  support  my  intended  study  and  research  des.gn. 

Smith,  M.  &  Glass,  G.  (1987).  n^rch  and  evaluation  in  education  W"«1 1"?  SOSM  Science 
New  Jersey:  Prentice-Hall. 

This  book  provides  a  nice  overview  of  qualitative  research.  The  authors  note  that  "the 
aitn  of  naturalistic  research  is  to  understand  the  persons  involved,  their  behave  and  P^P'™8; 
and  the  influence  of  the  physical,  social,  and  psychological  environment  or  conten  on  them 
SmiA  &  Glass.  1987,  p.  257).  Included  in  the  review  is  a  differentiation  between  quahtaUve  and 
quantitative  research. 

The  most  relevant  information  included  in  the  book  is  a  method  to  critique  qualitative 
research.  The  authors  note  that  because  of  the  variety  in  qualitative  studies  spec.fic  gu.dehnes 
cannot  be  assigned.  However,  certain  issues  regarding  evaluation  of  quahtaUve  research  can  be 
applied. 

These  issues  are: 

1          TW  .rv.nt  collecting  data  -  This  is  a  primary  control  that  needs  to  be  assessed. 
If  time  is  restricted  more  data  needs  to  be  collected  from  alternate  sources. 

2.  Access  to  data  -  The  quality  of  the  relationship  determines  the  ability  to  access 
data. 

3.  Naturalness  of  the  data  -  The  ideal  study  portrays  a  case  as  realistically  as 
possible. 

4.  p^rcher  self-criticism  -  The  researcher  should  identify  his  preconceptions  and 
biases. 

5.  T/>cical  validity  -  There  should  be  carefully  reasoned  connections  between  the 
descriptive  data  if  conclusions  are  drawn. 

6.  Confirmation  -  Systematic  efforts  to  check  hypotheses  with  alternatives  should 
be  made. 

7.  n^rintive  adequacy  -  Methods  of  conducting  the  study  should  be  adequately 
and  thoroughly  described. 

8.  Significance  -  The  study  should  address  a  theoretically  important  question  in  a 
manner  which  can  be  answered. 
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Hutching  S.  (1988)    Education  and  Grounded  Theory.  In  Sherman,  R.  &  Webb,  R. 

£Z  "*         ''"  ff1irr"TiTrm   rpr»'        "MllMMli    London:  The 

^  ,  ™J b00k  ""fists  of  ■  ^  «««ys  *at  address  different  aspects  of  qualitative 
research.  Essay,  mclude  topics  related  to  the  historical,  pWIosophic«reduc«tion.l  and 
biographical  aspects.  However,  the  article  by  Hutchinson  discusses  grounded  theory  "which  is 

ZT£  Z  .  ?enCn,ti0n'  rathCT  *"  <««overy  oFwS  the  world 

appear,  to  be  to  participants  and  through  an  analysis  of  those  perceptions,  of  the  basic  social 

IZZf^lT^V  °tSniZ*Z  W°rld"     3)"  <1988>  notealattdu^ 

ntSin^!?  y  g;0Unded  fl,e0iy  40  "^"gently  and  imaginatively  explore  the  social 

psychological  consequences  of  school  life"  (p.  127). 

code.  anJtSS"  ^T^6  f0f  C°ndUCtin8  11118  research-   The  researcher  collects, 

Se2fi^  ^,  •         ?  ? 1  ^  T*  ^  7116  '"P0*"**  Md  n^ner  °f  coding  the  data  i 
identified.   It  is  essential  to  understand  the  nature  of  data  collection  in  order  that  emeroinc 
concepts  can  be  identified  and  clarified.  emerging 

This  would  be  an  essentially  to  review  when  establishing  the  research  desien  "Rich 

sotoTd  If  T,*  ^  V*"****'  y™"*  •  final  Product  thTt  isThSetic^ 
sound  and  can  be  put  to  practical  use"  (p.  138).  y 

Eisner,  E.  (1991)    The  lightened  eve.  a^tlw  ,     irv  flnH  t|w  ,nh  f 
rtHCational  ffmrflff.  New  York:  Marian  Publishing  Co. 

thonoh.  ISV*  CX?1,ent  b°°k  WhiGh  haS  83  itS  emPhasis  11,6  relationship  between  qualitative 
thought  and  human  understanding.  The  author  thoroughly  analyzes  qualitative  inquiry  within  he 
contex  of  fte  educational  setting.  ^tqfU^i^iti^^Ml^'^^ 
trying  to  understand  what  teachers  and  children  do  in  the  settings  in  which  mey  3"  (p  H) 

Th,       ILTU,d  *  d,iffiCUU  10  ldentify     "P*01  0f  11,18  teJtt  which  Wou'd  P™ve  most  relevant 

SbSr^tt    ta,ive  reswch  in  a — ^ way- » — -* 

for  their  ^^^1^  ^  *"*  by  doCt0ral  studentS  u,i,i*ng  -"Chod 

i2ta2j^ST"  ^^-"PPO^^fi^ive  research  for  dis.serta«ions  which  investiga.e 

The  creation  of  new  visions  of  educational  inquiry  can  not  only  broaden  the 
ways  in  which  we  study  schooling,  but  even  more  important,  can  expand  our 
conception  of  human  cognition  and  help  us  develop  new  forms  of  pedagogical 

'^clfflsJSr^  "  NW  Y°rk:  Teachere  C°»<*<> 

collection^iThe*DU™ntialrreferenCe  ^  *"  ^  interviewing  as  a  means  of  data 

seSnnof  US,D8  m,emeWS  ^  dCSCribed  ',0ng  Wi,h  eStob,ishmenl  »nd  -rrec, 

exWl^SnTiS'  "ST^  ri,yiCJ°ndUCtCd-  11,eref0re'  ,he  ch»P^  on  techniques  is 
extremely  beneficial.   The  author  aclcnowledges  that  interviewing  skills  can  be  learned  and 
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developed.  The  concepts  of  listening  skills,  timing,  clarifying  and  appropriate  types  of  questioning 
are  all  discussed. 

The  author's  purpose  is  to  define  the  essential  elements  of  an  interview  in  order  that  the 
strength  of  this  technique  can  be  achieved.  'In-depth  interviewing'*  strength  is  that  through  it  we 
can  come  to  understand  the  detail  of  people's  experience  from  their  point  of  view  (p.  103). 

Bogdan,  R.  &  Biklen,  S.  (1992).  On-lifti™  research  for  education.  Boston:  Allyn  &  Bacon. 

This  book  serves  as  an  introduction  to  qualitative  research.  It  covers  the  historical 
evolution  of  this  type  of  research  and  proceeds  to  address  various  kinds  of  qualitative  research. 
One  of  the  nicest  aspects  of  this  book  is  the  author's  use  of  examples  to  illustrate  data  analysis. 

The  chapter  on  writing  up  the  research  is  very  helpful.  It  addresses  the  problem  of  how 
to  begin  through  getting  started  and  actually  completing  the  written  manuscript.  This  would  be 
especially  helpful  when  attempting  to  write  either  a  dissertation  or  research  for  publication. 

Goetz,  J.  &  LeCompte,  M.  (1984).  F.thnngranhv  and  Qualitative  fain  i"  edwfttivnnl 
research.  San  Diego:  Academic  Press,  Inc. 

This  book  deals  strictly  with  ethnographic  research  in  the  educational  setting.  It  begins 
with  a  detailed  description  of  educational  ethnography.  The  authors  note  that  "the  purpose  of 
educational  ethnography  is  to  provide  rich^descriptive  data  about  the  contexts,  activities  and 
beliefs  of  participants  in  educational  settings"  (p.  17). 

The  chapter  which  addresses  the  assessment  of  the  research  design  is  very  helpful.  A 
discussion  of  quality  control,  credibility,  reliability,  validity  and  evaluation  of  the  design  is 
included  in  this  chapter.  It  would  prove  very  useful  in  the  development  of  the  research  design. 

Eisner,  E.  &  Peshkin,  A.  (1990).  Qualitative  inquiry  in  education.  Thy  continuing  debate 
New  York:  Teachers  College  Columbia  University. 

This  book  includes  a  series  of  essays  that  address  the  relevance  of  qualitative  research 
in  education.  Most  of  the  articles  are  centered  around  the  topics  of  general izability  and  validity, 
since  these  are  aspects  of  qualitative  research  which  are  often  subject  to  criticism. 

However,  the  series  of  articles  concerning  ethical  issues  in  qualitative  research  is  of 
utmost  importance.  This  is  an  area  that  is  often  not  covered  comprehensively  in  a  standard 
textbook.  The  novice  researcher,  especially,  needs  to  be  aware  of  all  of  the  ethical  considerations 
which  must  be  addressed.  "...  there  is  a  multifaceted,  overlapping  complex  of  potential  ethical 
issues  in  every  dimension  of  their  work  that  they  need  to  become  sensitive  to  if  they  are  to  be  true 
to  their  commitment  to  a  human  science"  (p.  296).  It  is  essential  to  read  these  articles  before 
initiating  a  research  project. 

Strauss,  A.  (1987).  Qualitative  analv««  for  social  scientists.  New  York:  Cambridge  University 
Press. 

This  book  deals  primarily  with  the  analysis  of  qualitative  data.  The  one  area  that  is 
covered  more  specifically  in  this  book  is  the  use  of  integrative  diagrams  as  a  means  of  making 
data  come  together.  This  proves  useful  as  a  method  of  sequencing  data  in  order  to  analyze  it 
fully. 
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The  other  interesting  area  of  this  book  is  the  chapter  of  questions  and  answers.  This 

SSSSSIST t0  a  neophyte  researcher  il  ^  ™y <ues,ions  <h« 
spn^  ^^S^Sf^^        *  ■ """" 

This  book  addresses  the  concept  of  ethnography.   The  explanations  of  ethnographic 

EST  rqU<>;  ^  "U,b0rS  J*  eXpl4in  Ae  ideas  of  cultural  meaning,  experience  and 
description.  These  chapters  are  easily  understood. 

However,  the  most  relevant  part  of  this  book  is  the  series  of  ethnographic  case  studies 
JWov.de  the  reader  with  an  understanding  of  the  procedures  for  conducing  Zo^c 

P^.1"  S^'mU2-! *e  ptecf^  bibliography  covers  the  entire  realm  of  qualitative  research. 
wou7d^K^«  !h,%hlStonc-1  evolu,ion  10  *»  P™*«  *V  criticisms  has  been  addresS « 
Z 2?  £  b*°efic,al  for  "JOM  beginning  a  qualitative  research  project  to  review  the  books 
highlighting  the  aspects  specific  to  the  chosen  research  design. 
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National  Convention,  January  1993 
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Abstract 

This  study  sought  to  describe  graduate  students'  opinions  of  selected  leaminn 
SESf  TZUnt9L°d  in  ,ar^roup,  teacher-ledinstruZionT^ 
subjects,  forty  graduate  students  of  instructional  technology  at  Fmaior  western 
umversity  completed  a  questionnaire.  The  instrument  aske^subS^tofate 
22*  Lelntern? tiVi leamin9  actMties  alon9  *Ve  dimensions:  re^uencl  the 
leZZn  t^,00^6?  in!erest  in  the  actMty:  effectiveness  of  the  activity  as  a 

e^sZe  tth7Snnal  «P85/  °ffhe  actMty;  and  last>  desir*  for  7e7ter 
%£»ani  JJ.  °7* Results  indicated  that  lecture  is  the  most 
SSS^  ™.0O!'n*red  activ'fy-  *Me  also  the  least  motivating  and  effective 
Further,  subjects  indicated  a  strong  preference  for  areaterexoo^tn^t 

TrZ'  SST8  aCmiBS  ^  !spr0b,6m  ST-M 

Instructional  design  theory  and  models  stress  the  criticality  of  enaaaina  students 
"Jfl  af        requiring  overt  performance.   Current  consfructiv  st 
conceptions  of  the  learning  process,  however,  suggest  tha Tthose  ac Ss which 

X«vV^S,,0a  T*  SXiS,in9  knOW,6d9e  and  wS  be  mos 

ffi^'attc  ?haTd  ?aCh6rS  ar!  challen9ed  to  select  powerful,  varied 

SESSE  h,9her-order  ,earnin9  ou,comes  and 

ISaE^S  sludy,was  conducted  to  determine  which  kinds  of  learning  activities 
sTucS ^  U9eksrdelUS^fiHnd  m?St  m6anin9,ul'  motiva,in9'  and  eSe  The 

tSSST  op  s  re9ardin9  ,en  selec,ed  ieamin9 

Method 

Subjects 
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Instrument  and  procedure 


A  questionnaire  was  designed  and  pilot-tested  for  the  purpose  of  ascertaining 
subjects'  opinions  about  selected  learning  activities,  including  the  following: 
brainstorming,  problem  solving,  case  studies,  lecture,  structured  note  taking, 
games,  role  playing,  responding  to  written  questions,  responding  to  oral 
questions,  and  small-group  work.  The  activities  were  selected  because  they  are 
consistent  with  the  types  of  activities  employed  by  the  author  In  conducting 
graduate  courses. 

The  seven-page  instrument  began  with  a  cover  letter  explaining  the  Intent  of  the 
study  and  a  sample  item.  The  body  of  the  instrument  was  built  upon  a  consistent 
series  of  items  that  was  presented  for  each  of  the  ten  activities  in  question. 

The  format  for  each  learning  activity,  as  shown  in  Appendix  A,  was  as  follows:  to 
begin,  the  given  activity  was  identified  and  defined.  The  first  three  items  asked 
for  ratings  on  ten-point  scales.  The  first  item  asksd  the  subject  to  rate  how  often 
s/he  encountered  the  activity;  the  second  asked  for  a  rating  regarding  Interest  in 
the  given  activity;  the  third  item  asked  for  a  rating  of  the  effectiveness  of  the 
activity  as  a  iearning  tool.  The  fourth  item  asked  whether  the  subject  found  the 
given  activity  motivating  and  required  a  "yes/no"  response.  It  was  followed  by  a 
space  for  comment.  The  fifth  item  asked  if  the  subject  would  like  to  do  the  given 
activity  more  often.  Finally,  a  prompt  to  provide  written  suggestions  for 
improving  the  given  activity  was  provided.  At  the  end  of  the  instrument, 
subjects  could  fill-in  their  own  learning  activities  (not  specified  by  the  author) 
and  answer  the  same  series  of  questions  for  that  activity. 

The  instrument  was  administered  to  two  intact  sections  of  an  introductory 
instructional  design  (ID)  seminar  and  to  one  intact  section  of  an  advanced  ID 
seminar.  The  researcher  began  by  briefly  describing  the  purpose  of  the  study 
and  by  guiding  subjects  through  completion  of  the  sample  item.  Subjects 
completed  the  instrument  in  silence  over  fifteen  minutes  prior  to  the  start  of 
their  regular  classes.  Forty  subjects  responded  to  the  questionnaire. 

Results 

Table  1  shows  the  means  and  standard  deviations  for  the  frequency  ratings  by 
activity.  Lecture  received  the  highest  mean  rating  of  9.6  in  terms  of  frequency 
while  games  and  role  playing  tied  for  the  lowest  rating,  with  mean  ratings  of  4.4, 
respectively. 
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Table  3:  Mean  Ratings  of  Effectiveness  as  Learning  Tool 

Activity 

Mean 

OlanOaiU  UcVlatlOM 

Brainstorming 

8.0 

1.9 

Problem  Solving 

8.4 

1.6 

Case  Studies 

8.5 

1.4 

Lecture 

5.9 

2.0 

Structured  Notes 

7.5 

2.3 

Games 

7.2 

2.3 

Role  Plfavs 

7.2 

2.2 

Discussion/Prep'd  Ques. 

7.3 

2.2 

Oral  Questions 

6.7 

2.5 

Small  Grp  Work 

8.2 

1.7 

Table  4  shows  the  proportions  of  the  40  subjects'  responses  to  questions 
concerning  whether  they  found  a  given  activity  motivating  and  whether  they 
wished  to  "do  the  activity-  more  often.  The  "ns"  option  stood  for  "not  sure"  for 
those  who  were  non-committal. 

Results  indicate  that  subjects  find  problem  solving  and  case  studies  to  be  the  most 
motivating  of  the  activities  considered.  A  proportion  of  .825  identified  each  of 
these  as  motivating.  Conversely,  subjects  indicated  that  iecture  was  the  least 
motivating  activity  at  a  proportion  of  .35.  However,  a  proportion  of  ,375 
expressed  uncertainty  as  to  whether  or  not  they  found  a  lecture  to  be  motivating. 

Subjects  further  indicated  a  preference  for  more  problem  solving  activities,  at  a 
proportion  of  .775.  Other  activities  to  which  they  would  like  more  exposure 
included  case  studies,  brainstorming,  and  small  group  work. 

Items  at  the  end  of  the  instrument  allowed  subjects  to  identify  other  learning 
activities  not  presented  by  the  researcher  and  to  answer  the  same  series  of 
question  posed  for  the  given  activities.  Respondents  identified  student 
presentations,  feedback  sessions  with  other  students,  individualized  activities, 
independent  study,  field  trips,  expert  lectures,  simulations,  and  multimedia  or 
computer-assisted  presentations  as  other  viable  types  of  learning  activities. 

Discussion 

The  present  study  revealed  that  the  surveyed  graduate  students  do  have  definite 
opinions  regarding  the  motivational  appeal  and  effectiveness  of  various  kinds  of 
learning  activities.  It  is  interesting  to  note  that  while  subjects  found  lectures  to 
be  unmotivating  and  ineffective,  this  type  of  activity  is  the  one  they  most 
frequently  encounter  in  their  graduate  courses.  It  seems  safe  to  say,  further, 
that  lecturing  is  not  an  :  eal  means  of  furthering  knowledge  construction  by 
students. 
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Table  4:  Proportions  of  Motivational  and  Repetition  Opinion*  by  Activity 

Activity 

Is  it  motivating? 

Do  more  often? 

ves     no  ns 

_ves     no  ns 

Brainstorming 

.80 

.10 

.10 

.675 

.125 

.20 

Problem  solving 

.825 

.025 

.15 

.775 

.075 

.15 

Case  studies 

.825 

.05 

.125 

.75 

.10 

.15 

Lecture 

.275 

.35 

.375 

.175 

.625 

.20 

Structured  note  taking 

.50 

.325 

.175 

.45 

.35 

.20 

Games 

.75 

.125 

.125 

.575 

.225 

.20 

Role  playing 

.60 

.20 

.20 

.525 

.275 

.20 

Written  questions 

.625 

.20 

.175 

.575 

.225 

.20 

Oral  questions 

.525 

.35 

.125 

.375 

.425 

Z0 

Small  group  work 

.775 

.10 

.125 

.70 

.20 

.10 

Subjects  indicated  a  strong  desire  for  more  challenging  learning  activities  such 

ZS^llVT^'  SaSe,S,udies'  ro,e  P'3*^  and  brLstorming  Findfngs  will 
?sJ?  !  author  in  developing  instructional  strategies  that  incorporate  actives 
hat  are  both  motivating  and  effective.  Other  s  working  with  araduat e  stuXs « 
instruct^  design  or  technology  are  encouraged  to  employ  ?he  qu?stfon  fo?ma? 
presented  m  Appendix  A  toward  determining  their  own  students"  prlfe  ences 
Future  stud.es  of  this  sort  will  be  more  valuable  as  they  engage  greyer  numbers 
of  subjects  from  diverse  institutions  in  an  effort  ,o  fer  e,  out'mo're  generXa^ 
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How  ofu*  have  you  encountered  this  activity? 

^  1     2     3     4     S     6     7     8     9     10  Qftcn 
Rate  your  totewt  level  In  brainstorming  activities. 

1     2     3     4     5     6     7     8   .9     10  iiiah 
How  do  you  rat.  brainstorming  In  terms  of  efltoW.  as  a  learning  tool? 

1     2     3     «     S     6     7     8     9     10  dlflh 

Do  you  find  brainstorming  activities  woe/v.^?  Yes_  No_  Not,ure 
Why  or  why  not?   


Would  you  like  to  do  this  type  of  activity  monoftuil  Ye,  _  No  _  Notfur, 
List  on  the  back  s-wsttow  you  have  for  improving  a  brainstorming  activity. 

*50  ad 

w  o  *w 


Tide: 

Digitized  Speech  as  Feedback 
in  Computer-Based  Instructio] 


Authors: 

Gregory  C.  Sales 
Michael  D.  Johnston 


851 


Digitized  Speech  u  Feedback 
AECT1993  2 


Digitized  Speech  as  Feedback 
in  Computer-Based  Instruction 


j.   We  should  not  be  surprised  by  the  flood  of  educational  products  utilizing 
digxbzed  speech  currently  moving  through  the  home  and  school  markets.  From 
the  time  of  Socrates,  speech  has  been  an  important  part  of  instruction  For 
most  of  history  speech  has  been  the  primary  mean  of  communicating  content 
to  the  learner.  Only  recently  has  text  in  the  form  of  books  and  other 
technologies  been  effectively  utilized  to  increase  the  number  of  learners  that 
can  be  reached  and  to  provide  a  consistent  delivery  of  information.  Naturally 
then,  the  integration  of  speech  into  instructional  software  has  great  intuitive 

aPwu-!Spl     y   i4      1x5  Unked  to  text»  graphics  and  animation. 

While  the  phenomenon  of  talking  software  may  now  be  upon  us  in  full,  it 
has  been  evolving  for  years.  Several  techniques  for  generating  computer 
speech  have  existed  since  the  1970s.  However,  these  early  techniques  were 
seldom  utilized  because  of  cost,  equipment  requirements  or  quality  issues 
Kecent  advances  in  hardware  and  software  have  made  the  use  of  digitized 
speech  a  practical  option  for  many  instructional  applications 

Software  designers  and  developers  appear  to  believe  that  speech  will  make 
computer-based  instruction  (CBI)  more  attractive,  more  effective  and 
ultimately  more  marketable.  They  are  rushing  to  integrate  speech  into  a 
varietyof  products.  Their  motives  for  applying  this  technology,  however,  may 
be  based  purely  on  consumers'  demonstrated  eagerness  to  see,  use  and  own 
the  latest  innovations. 

There  is  little  empirical  research  to  indicate  that  educational  gains  can  be 
achieved  by  simply  incorporating  speech  in  computer  software.  Decades  of 
empirical  research  on  the  use  of  sound,  music  and  speech  in  instruction 
h    frS1;,0^9?  Gumming  &  McCorriston,  1981;  Fleming,  1980; 
Haugh,  1952;  Rulon  &  Others,  1943;  Rysavy  &  Sales,  1991;  Sanders, 
Benbasset  &  Smith,  1976)  provide  only  limited  guidance  on  how  these  features 
might  be  used  in  computer-based  instruction.  This  guidance  falls  well  short  of 
^entifying  when  speech  is  an  appropriate  component  of  CBI,  the  role  it  should 
perform  or  the  interface  and  operational  support  that  a  program  should 
provide  to  ensure  optimal  learning. 

Many  questions  that  may  inform  decision  about  the  utilization  of  speech  in 
[emT  unaJnswered-  This  Paper  reports  on  two  studies  conducted  in 
an  effort  to  help  use  better  understand  the  role  of  digitized  speech  as  feedback 

SMlchaelDUl&)th18      °frcSearCh  iS  ***  rep0rted  at  ***** 

r  J?"?^  1116  fir8t  8*udy  examined  the  use  of  familiar  and  unfamiliar  voice 
feedback  in  two  computer-based  lessons  designed  to  teach  advertising 
techniques  (propaganda  techniques)  to  fifth  graders.  The  role  of  the  voice 
Z *  P™£  *^tedBe.  of  response,  guidance  and  motivation  during 
the  practice  portion  of  the  instruction.  (See  figures  1-6  for  examples  of  the 

ZTl  f^l^u  *?e  ^P^r-based  lesson,  the  feedback  types,  and 
content  of  the  feedback  messages.) 

The  primary  research  questions  addressed  in  this  study  were: 
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•  Will  the  use  of  a  feedback  provider  whom  learners  know  influence  their 
performance  in  the  instruction,  as  measured  by  the  posttest.' 

•  Will  the  use  of  (1)  voice  with  text  or  (2)  voice  with  a  speaker's  image 
influence  the  learners'  performance  in  computer-based  instruction.'' 

•  Are  there  differences  (e.g.,  use  of  optional  buttons,  achievement,  time 
to  complete)  in  performance  that  can  be  attributed  to  gender.' 

Studv_2  The  second  study  examined  the  effects  of  three  types  of  feedback 
(1)  spdkenaudio  only,  (2)  printed  and  read  (spoken),  and  (3)  spoken  by  an 
animated  character.  These  feedback  types  were  substituted  m  the 
propaganda  software  used  for  the  first  study  (See  figures  7-9  for  espies  of 
SieWeen  displays  from  the  computer-based  lesson,  the  feedback  types,  and 
content  of  the  feedback  messages.) 

The  primary  research  questions  were: 

•  Will  the  type  of  feedback  (i.e.,  form  of  presentation  -  (1)  spoken  audio 
only,  (2)  printed  and  read  (spoken),  and  (3)  spoken  by  an  animated 
character)  effect  performance? 

•  Will  reading  ability  interact  with  performance  as  measured  by 
posttest  scores? 

UteWhatCis  thereof  speech  in  instruction?  It  would  be  difficult  to  deny  that 
speech  can  play  a  central  role  in  the  delivery  of  instruction.  Most  modern 
models  of  instruction  assume  the  extensive  use  of  spoken  or  written  word* 
For  example,  each  of  the  events  of  instruction  (Gagne,  Bnggs  &  Wager,1992), 
from  informing  the  learner  of  the  objectives  to  enhancing  retention  and 
transfer,  can  be  accomplished  at  some  level  through  spoken  words. 

The  power  and  influence  of  the  spoken  word  is  equally  difficult  to  deny,  lne 
effectiveness  and  impact  of  speech  in  such  roles  as  the  delivery  of  content, 
feedback  and  motivation  in  classroom  instruction  are  well  documented  te.g., 

Brophy,  1981, 1986).  it_     _  <. 

Salomon  (1979, 1985)  argues  that  research  on  the  effectiveness  of  media 
must  look  at  the  most  essential  characteristics  of  symbol  systems  being 
utilized.  In  software  that  utilizes  digitized  speech,  Brophy  s  work  on  the  impact 
of  teachers'  comments  indicates  that  these  essential  characteristics  are  the 
familiarity  of  the  source,  the  nature  of  the  learners  relationship  with  the 
source,  and  the  attitude  communicated  through  tones  and  inflections  m 
feedback  to  the  learner.  ,    ,        ....  . 

What  do  we  know  about  the  effects  of  speech  (and  other  audio)  in  computer 
software?  Researchers  have  expressed  concern  over  techniques  and  strategies 
for  the  use  of  speech  in  CBI.  Sanders,  Benbasset  and  Smith  (1976),  for 
example,  argue  that  for  speech  to  add  significantly  to  earning,  it  mus it not 
"merely  take  the  place  of  a  hard-copy  manual  or  of  printed  text  on  a  computer 
terminal."  For  example,  one  method  of  adding  value  to  the  use  of  audio  might 
be  to  personalize  the  software.  Personalization  strategies  have s  been  shown  to 
increase  the  effectiveness  of  CBI  (Ross,  1983,  Ross  &  Anand,  1987). 
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Research  on  the  use  of  speech  in  CBI  has  had  mixed  results.  Research 
conducted  with  young  children  learning  phonics  and  vocabulary  (Sales  & 
Johnson,  1991)  found  no  achievement  or  attitude  differences  for  students  in 
several  freatments  involving  different  levels  of  speech  and  "motivational" 
sound.  The  authors  suggest  several  explanations  including  the  low  level  of 
students'  metacognitive  skills,  and  the  redundant  presentation  of  critical 
information  through  text. 

Two  research  studies  examining  the  different  uses  of  audio  in  computer- 
based  instruction  were  conducted  with  special  needs  students  (Chiang,  1983 
Wiener,  1991).  These  studies  had  dramatically  different  findings.  Chiang  found 
audio  designed  to  reward  and  motivate  students  was  actually  detrimental  to 
learmng.  He  argued  that  audio  served  as  a  distractor,  causing  students  to 
require  more  study  time  to  achieve  the  same  level  of  learning  Wiener 
however,  found  significant  achievement  effects  when  using  computer  ' 
generated  speech  to  teach  word  recognition  to  mentally  handicapped  junior 
high  school  students.  J 

In  summary,  speech  can  be  a  critical  component  of  instruction.  It  appears 
to  be  particularly  valuable  as  a  means  of  communication  between  the  teacher 
and  the  learner.  Much  of  the  communication  results  from  when  and  how  the 
message  is  delivered.  These  characteristics  of  the  discourse  may  be  essential 
to  the  learning  outcomes. 

Speech  and  other  auditory  elements  of  CBI  have  had  mixed  success  in  a 
number  of  research  studies.  Our  limited  understanding  of  these  findings  is  in 
part  due  to  the  small  amount  of  research  that  has  been  conducted  This  in 
turn,  can  be  traced  to  the  rapid  changes  technologies  and  the  fact  that  the 

Stu^One  °f  8Pe6Ch  in  CBI  ha8  0nly  recentlv  become  Practical. 

100  fifth^Sf*;  ?iefsu^ects  whu°  voluntarily  participated  in  this  study  were 
??u    *  j    x  stotenU  from  a  suburban  Minneapolis  elementary  school.  All 
tile  study  COntaCt  With  comPuter-based  instruction  prior  to 

Instructional  materials.  The  software,  which  provided  instruction  on 
propaganda  techniques  used  in  advertising,  is  based  on  the  software  used  by 
Carrier  and  associates  (see  Carrier  &WMiams  1988;  Carrier,  Davidson  and 
Yviwams,  1985).  The  new  version,  was  designed  to  take  advantage  of  the 
graphic  and  sound  capabilities  of  Macintosh  computers  with  monochrome 
monitors  It  was  created  using  HyperCard  2. 1  and  for  this  study  it  was  run  on 
Macintosh  LC  computers  in  a  thirty  station  lab. 

During  the  data  collection,  all  of  the  participants  were  required  to  wear 
neadphones.  The  software  and  individual  student  records  were  stored  at  each 
workstation  on  the  hard  drive.  At  the  end  of  each  day  of  data  collection,  the 
student  records  were  collected  and  aggregated  for  analyses 
,JZS'i?-         design  was  used  (Figure  10)  in  this  research.  Variable  one, 
degree  of  famihanty  with  the  feedback  provider,  has  two  levels  -  familiar  and 
SlSSS"'  ?!  t^j£F*+ type  of  feedback,  has  two  levels  -  Voice  Only 
™iV?*™th  Character.  The  treatments  in  the  familiar  condition 

used  a  person  of  authority  (the  school  s  media  center  teacher/librarian)  that 
was  known  to  the  subjects  as  the  vehicle  for  delivering  feedback    In  the  « 
treatments  m  the  unfamiliar  condition  a  fictitious  character  unknown  to  the 
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subjects  was  used.  In  addition  to  the  voice  feedback,  two  groups  received  a 
visual  of  the  speaker's  face.  Animation  techniques  were  used  to  create  the 
appearance  that  the  speaker  was  actually  talking  as  the  feedback  was 
delivered 

Procedures.  Subjects  participated  in  the  research  in  intact  classroom 
groups  but  individuals  within  groups  were  randomly  assigned  to  one  of  four 
treatments.  Classroom  teachers  did  not  play  a  role  m  the  delivery  of  the 

UWtSJer  a  period  of  three  days,  each  subject  completed  two  computer-based 
lessons  on  techniques  used  in  advertising.  The  first  lesson  which  lasted 
approximately  20  minutes  introduced  students  to  four  techniques  and  then- 
defining  attributes.  Students  were  given  an  opportunity  to  practice  identiryins 
examples  and  non-examples  of  two  of  the  techniques  introduced.  The  second 
lesson,  which  lasted  approximately  30  minutes,  provided  practice  on  the 
remaining  two  techniques  and  allowed  for  a  brief  review.  Following  the  second 
lesson  the  students  completed  an  on-line  test  consisting [of  25  items 
(Cronbach's  alpha,  r=.87),  five  representing  each  of  the  four  techniques  studied 
and  five  non-examples.  Finally,  students  responded  on  paper  to  seven 
questions  focusing  on  effort,  attitude  and  learning. 

Data  Analysis  and  Results.  Analyses  of  variance  were  conducted  to 
determine  the  effects  of  the  treatment  variables  on  the  posttest  scor  . 
Neither  the  familiarity  of  the  voice  nor  the  presence  of  a  feedback  provider 
influenced  overall  performance  (F=.54,  p=.462  and  F=.39,  p=.321,  see  Table  IX 
No  interactions  of  the  treatment  variables  were  found.  Gender  was  found  to  be 
significant  with  girls  outscoring  boys  by  an  average  of  2  1/2  points.  Means  for 
the  two  groups  are  shown  in  Table  2. 

Discussion.  Analysis  of  the  data  related  to  the  primary  research 
questions,  which  addressed  the  issues  of  familiar  versus  unfamiliar  and  voice 
versus  animated  speaker,  found  no  differences.  This  may  indicate  that  the 
attribution  of  authority  does  not  transfer  to  a  computer  representation  of  a 
teacher  Or,  it  may  mean  that  a  student's  sense  of  accountability  in  a 
computer-based  lesson  is  not  influenced  by  the  attribution  of  feedback  to  a 
specific  source.  Regardless  of  the  source  of  the  feedback  as  represented  in  the 
software,  students  may  simple  associate  the  feedback  with  the  inanimate, 
non-judgmental  delivery  technology. 

Some  issues  of  secondary  interest  were  identified  and  are  deserving  ot 
discussion.  Use  of  the  "Say  it  again."  option,  which  allowed  students  to  have 
the  verbal  feedback  repeated,  appeared  to  be  related  to  gender  and  treatment. 
Boys  tended  to  use  this  option  more  in  the  familiar/voice  with  text  treatment. 
Girls  tended  to  use  this  option  more  in  the  unfamiliar/voice  with  face 

treatment.  ,     „        .  „  ,  .      ..  c.u 

Another,  and  perhaps  related,  secondary  finding  is  that  in  spite  ot  the 
pattern  of  option  button  use,  girls  spent  more  time  working  through 
treatments  in  which  the  feedback  provider  was  known.  Boys  spent  more  time 
in  treatments  where  the  feedback  provider  was  unknown.  These  findings 
might  indicate  that  girls  were  willing  to  work  harder  at  studying  the 
information  on  the  screen  when  they  knew  the  feedback  provider.  When  the 
provider  was  unknown,  they  may  not  have  tried  as  hard.  The  feedback 
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provider  m  the  software  studied  was  a  female.  This  may  also  account  for  the 
differences  m  the  ways  in  which  the  boys  and  girls  used  the  material? 
fil       a  re?ult  ofthese  findings  a  second  study  was  designed  and  conducted  to 
father  explore  the  use  of  speech  as  feedback.  It  was  determined  SfoT&e 
second  study  we  would  use  a  male  speaker  was  unknown  to  the  learners  Two 

r01'6  only  311(1  text       was  read,  were  added  to  isdate  any 
effect  attributed  to  the  voice  variable  16 

Study  Two 

m-  *u°rtifpaIU!'  Su^ecta  volunteering  to  participate  in  this  study  were  145 
^SS^J^?       fbur^n  Minneapolis  middle  school  thVserad  only 
fifth  and  ^xth  grades.  Subset  of  volunteers  with  high  (N=41)  or  low(N=49) 

SSteK^r  uiden^ 0nlv fl™ these mdivid^were 
^dy  ^thou«h  811 145  Participated  in  the  instruction.  All  of  the 

Z^Zd^  C°ntaCt       COmPuter-ba8ed  instruction  prior  to 

Instructional  Materials.  The  instructional  materials  used  in  this  lesson 
were  the  same  as  those  describe  in  Study  1  except  for  the  ddiv^f  feedback 

d^etdTS?  to      treatmen?  COndition-  ™e  feedback  *  S s  nay 
differed I  in  that  the  audio  was  a  male  voice  and  the  animated  figure  was  of  an 
unfamiliar  male  cartoon  character. 

The  computers  used  in  this  study  were  in  two  labs  of  approximately  15 

^^^^ 
has&i%^ 

was^red^  dass™m  groups  were  divided  in  half  and  each  half 

was  required  to  report  to  one  of  two  Macintosh  computer  labs  where  the 
studies  were  conducted.  In  each  lab  the  subjects  were  randomlv aliened  to 
Z l^oT^  ClaS8ro°m  n0t       a  -StoeTltry  of 

Data  Analyst  and  Results.  Analyses  of  variance  were  conducted  to 
^^l**  effeCt|  °f  ?e  ^atment  variables  on  tlT,£SX^W  Type 

a  J^TTn\^?aly8,i8  otthe  ^ta  related  to  the  primary  research  Question 
type  of  feedback)  found  no  difference.  However,  there  w^rc^nsTstent 

^r  8tudente  ^ceiving  only  voice  feedback  to  s^ore  Wghe on  L 
posttest.  This  may  indicate  that  supplementing  the  audio  with  text  or 

"^ftSJ?      1faJner8'  a^nti0n  re8ultine  in  declsS^mprehension 
abilito  to^fSfl lated  t«  reading  ability  appear  to  reflect  the  participate 
tWrti?  "S*  ^Jf"?  6,0111  ^  text  in  the  lesson.  These  findings  indicate 
p"  ^  d°e8  -  — te  ^  Ste^ 

Conclusions  and  Recommendations 

rp«»uf  °fTi,rig  "!  *  number  °/ conclu8ions  and  recommendations  based  on  the 
results  of  these  studies  and  the  observations  of  the  researchers.  wSiey 
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may  be  tentative,  they  none-the-less  provide  food  for  thought  and  direction  for 
future  work. 

1  Whose  voice  is  used  in  the  delivery  of  feedback  in  CBI  seems  to  be  of  little 
*  importance.  Perhaps  of  more  importance  is  the  gender  of  the  speaker.  In 

our  first  study,  where  a  female  agent  was  used,  female  students 
outperformed  male  students.  In  our  second  study,  where  &e  agent  was <  a 
male,  no  performance  difference  based  on  gender  was  found  Whether  this 
indicates  that  (1)  girls  work  harder  for  a  female  agent,  or  (2)  boys  don  t 
work  as  hard  for  a  female  agent,  is  worthy  of  further  investigation. 

2  The  image  associated  with  the  speaker  is  of  little  importance.  The 

'  students  in  our  study  did  not  appear  to  associate  an  agent  on  the  screen 
with  the  person  being  represented.  A  possible  topic  for  further 
investigation  is  whether  younger  learners  would  attribute  more  authority 
to  the  computer  agent  that  represented  a  person  they  know. 

3  It  is  reasonable  to  assume  that  if  speech  is  going  to  effect  a  change  in 
learning  outcomes,  then  the  amount  and  positioning  of  the  speech  within  a 
1  sson  are  of  critical  importance.  In  the  studies  we  conducted  speech  was 
used  only  to  deliver  feedback.  Much  of  the  content  was  delivered  through 
text  In  our  second  study  students'  reading  ability  was  found  to  have  a 
significant  effect  on  their  learning.  Therefore,  it  is  reasonable  to  assume 
that  an  effective  use  of  speech  technology  would  be  to  help  students 
overcome  reading  limitations  that  influence  their  processing  of  content. 

4  During  the  research  teachers  and  school  staff  repeatedly  commented  on 
'  the  effect  headphones  had  on  student  behavior.  Once  the  headphones 

were  on,  student  focused  their  attention  on  the  computer 'lesson.  Normal 
computer  lab  behavior  appears  to  involve  talking  with  others  and  looking 
around  the  room.  Whether  this  auditory  isolation  produces  better 
concentration,  and  thus  improved  performance,  is  uncertain. 

5  Another  observation  made  during  the  studies  was  that  students  in  the 
animated  character  treatments  were  more  demonstrative.  They  showed 
more  facial  expressions  and  talked  back  to  the  computers  more  than 
students  in  other  treatments.  The  observers  felt  these  students  seemed 
to  be  more  involved  and  to  enjoy  the  lessons  more.  This  may  indicate  a 
particular  value  to  this  method  of  integrating  speech. 

6  Regardless  of  what  our  research  finds.we  may  not  have  a  choice  but  to 
use  speech  and  agents  in  instructional  software.  The  use  of  these 
techniques  is  similar  to  the  use  of  color.  Increasingly  our  clients  expect 
products  to  contain  certain  features.  When  they  are  not  present, 
questions  are  raised  and  instructional  products  are  devalued.  Perhaps  the 
most  we  can  hope  to  do  at  this  time  is  to  moderate  what  we  believe  we 
can  accomplish  through  the  use  of  speech  technology  and  proceed 
cautiously  with  its  use. 
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7.  In  multimedia  environments  identification  of  primary  symbol  systems 
and  the  most  essential  characteristics"  of  each  is  no  small  task.  For 
spoken  words,  the  list  of  characteristics  includes,  but  is  not  limited  to- 
gender  of  the  speaker,  tone,  mode,  pacing,  cultural  cues,  and  message 
*  urthermore,  these  characteristics  may  have  varying  degrees  of 
importance  to  different  learners. 
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1  ADVERTISING         .w.  .u'T.m  u  ; 

While  you  are  working  on  this  lesson 
u«  Burton  be  te (Una  vou  how  you  are  9 
doing.  Be  sure  to  wear  your 
earphones! 

tf  you  want  to  hear  Ms.  Burton  say 

something,  )usi  press  the  'Say  tt  ■ 
j  again..*  button.  Try  it  now. 

When  you  have  finished  the  lesson,  you 
'  w«  be  given  a  test  to  check  how  much 
i  you  learned  about  these  four 

techniques.  ^ 

1 

1 M  U 

Figure  1.  Introduction  from  Propaganda  Techniques 
in  Advertising  lesson  describing  audio  controls. 


ADVE RTISING  banowagon  i i 


No*  try  this  one: 


lAinnesotans 
choose  UARTS 
CHERRY  PEE 
crcry  timet 


Fieure  3.  Typical  practice  item  from  the  "familiar'1 
\  oice  and  actor  treatment  just  prior  to  response 

checking 


ADVERTISING 


Now  try  this  one. 


Uh-hj*. 


\  ._ 

The  toy  to  the  comet  vmmHtox 
the  famous  person  b  only  shown  w*h 
the  dor*  thai  Isadveittoed.  He  1$ 
NOT  toftng  yoM  todrtr**»  pioduct 
This  IS  an  examp*  of  Transfer. 


Figure  5.  Feedback  as  displayed  to  the  voice  only 
treatments  (both  "familiar"  and  "unfamiliar'1  voice). 


ADVERTISING 


BAN P  WAGON    7 ' 


PRACTICE 
TIME 


Here's  a  chance  for  you  to  practice 
spottng  when  the  BanoVagon 
technique  H  used  In  ads. 
Ms.  Stranoberg  will  help  you. 

When  you  are  asked  a  question,  cttckc 
on  the  best  answer  that  appears  here. 

Then  dick  here  to  check  you  answer^ 


Figure  2.  Beginning  of  practice  section  explaining  re- 
sponse options. 


ADVERTISING  bandwagon  11 ; 


Now  try  this  one: 


Minncsotans 
choose  MARTS 
CHERRY  PTE  x 


words  tt»  -WOnnesotans,  moms, 
and  kids*  refer  to  a  LOT  of  people. 
You  ate  persuaded  to  buy 
scathing  Just  because  a  tot  of 
these  people  do.  So  thb  ad  does 
use  the  Bandwagon  technique. 


Figure  4.  Additional  feedback  appearing  in  an  ov  j 
window  after  audio  feedback  is  delivered. 


ADVERTISING  nuNSfEfl 


kJlL'ii«  W:  f^rvirrpion)  and  OIL  T  CO».L 


Look  *ery  closely  at  vhrs  ad 

i 


Figure  6.  Practice  item  using  the  "unfamiliar"  voice 
and  actor. 
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ADVERTISING  ustwdkim  -  9  I 


TfytWsoot.  Dotsthbatf  uwttw 
TESTIMONIAL  technique? 


•We  love 
Oscar  Mayer 
hot  dogs!" 


Figure  7:    Spoken  only. 


ADVERTISING 


Look  at  tread: 


BAMOWAGON-fc 


T  KIDS  LOVE 

*  RAISINS. 

?   '  m 

'j£  Tta  n  v.  Dolhin^  artificial 

in  a  raiiio';  IiIXJp  body 


as 


Is  tnrt  aneiampJe  of  Bandwagon? 


unted  end  read  (spoken). 


ADVERTISING 


!  ;  Trythrtor« 


Laa  year,  NAPA  sold  two  xmo 
parts  for  every  maa, woulml, 
child. .  .jtnd  dog  in  America. 


Does  thts  ad  u$e  the  BanoVaoon 


rirjure         Spoken  by  en  arr^^z^c  character 
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Table  1 

Analysis  of  Variance  -  Tests  of  Significance  for  Posttest  Scores 


Source  of  Variation 

within  cells 
covariate 

familiar/unfamiliar 

voice/image 

gender 

familiar/unfamiliar  by  voice/image 
familiar/unfamiliar  by  gender 
voice/image  by  gender 
familiar/unfamiliar  by  voice/image  by 
gender  


MS  F  SigofF 


25.88 

785.20 

30.34 

.00 

14.10 

.54 

.46 

25.71 

.99 

.32 

176.72 

6.83 

.01 

.64 

.02 

.87 

18.88 

.73 

.39 

1.35 

.05 

.82 

3.34 

.13 

.72 

Table  2 

Summary  of  Posttest  Scores  by  Gender 


Gender 
Boys 

Girls 


Mean 
Count 

Standard  Deviation 

1 

12.68 

47 

5.56 

2 

15.26 

53 

5.89 

14.05 
100 
5.85 


9 
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Figure  10 

2x2  Research  Design,  Cell  Size  and  Gender 


Familiar 

Unfamiliar 

Voice 
(with  text) 

22 

(14  girls.  8  boys) 

24 

(17  girls,  7  boys) 

Voice  with 
Animated  Character 

29 

(11  girls,  18  boys) 

25 

(11  girls,  14  boys) 

:rlc 
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Tabic  3.  Means  and  Standard  Deviations  for  Posttest  Scores  (Treatment  by  Ability) 


Treatments 

Ability 

1 

2 

3 

Total 

High 

Mean: 

SD: 

N: 

21.87 
3.83 
15 

22.11 
2.57 
9 

21.59 
4.00 
17 

21.80 
3.59 
41 

Low 

Mean: 

SD: 

N: 

16.07 
4.53 
14 

13.95 
6.73 
19 

15.13 
5.57 
16 

14.94 
5.74 
49 

Combined 

Mean: 

SD: 

N: 

19.07 
5.06 
29 

16.57 
6.87 
28 

18.45 
5.77 
33 

18.07 
5.95 
90 

Table  4.  Two-way  ANOVA  Results  for  Posttest  Scores  (Treatment  by  Reading  Ability) 

Source 

df 

SS 

MS 

F 

P 

Treatment 

2 

12.38 

6.19 

0.25 

.778 

Ability 

1 

985.60 

985.60 

40.14 

.000 

Interaction 

2 

19.47 

9.73 

0.40 

.674 

Error 

84 

2062.37 

24.55 
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social  sciences  in  recent  years,  based  on  the bel 3 ?tha ItSSeTtoiSd 

?976  te™fy  TflUrCC  th8ir  effective  and  innovative  use  ^  £X L 
Zr^T  admimstered  a  survey  to  American  adults  to  determine  thei 
&w^n,Aand  ^"^^ding  of  computers.  He  found  for  JSmoIe  that 
S„f     Amencans  he  surveyed  felt  that  computers  would  faEve 
education  .Researchers  interested  in  education  have  carted  out Ear 
Stad.es  with  teachers  and  students.  Lichtman  found'foTexamnle  that  in 
impL^onthoSr  S°meWhat  aPPrehen8ive  *«"  coWu^sTnd'th^  " 

and  Son ,  MQft«f  f  ^^Ple'  Ma-d8en  and  Seba«tiani  (1987  and  Berger 
attitudli Til      found  that  participation  in  computer  courses  influenced 

more  negative  ■Sif.JSA*        }'  8ome.8ay  8uch  experience  can  produce 
"uoy  For  instance"  in  a  recent 

students,  thus  preserviS ™SH„„vi         *at/°r  und«»Braduata  business 

»me«S'S2^lVU?d"  T"!*  C»">1»'««™        been  studied  in  . 

ou2SPflreh!T8iVf  8tUdieS  TeCent1^  "SSffihS ^based"?  HnftW  " 
questionnaire  development  on  thorough  reviews  of  the  literature, 
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administered  many  attitudinal  items,  and  have  subsequently  subjected  their 
items  to  factor  analysis  to  sort  through  the  actual  elements  of  computer 
attitudes  (cf.  Abdel-Gaid,  Trueblood  &  Shrigley,  1986). 

The  present  study  was  conducted  with  several  purposes  in  mind:  1) 
to  investigate  in  depth  how  attitudes  among  preservice  teachers  are 
influenced  by  participation  in  a  computer  literacy  course,  and  2)  to  test  the 
utility  of  an  extensive  attitude  questionnaire,  based  on  several  recent 
comprehensive  studies. 

Method 

The  subjects  were  75  college  preservice  teachers,  56  females  and  19 
males,  enrolled  in  two  five-week  summer  sessions  of  a  required  course  on 
computer  applications  in  education  at  a  m^jor  university  in  the  southwest. 

The  computer  literacy  course  was  systematically  developed  to  teach 
the  use  and  understanding  of  computer  technologies  in  an  educational 
setting.  The  course  is  taught  by  many  instructors,  in  a  multi-section  format, 
but  content,  activities,  projects  and  exams  are  the  same  for  all  sections.  All 
students  thus  received  the  same  instruction. 

As  a  normal  procedure  in  the  computer  literacy  course,  students  every 
semester  complete  pre-and-post  course  surveys  to  help  instructors  maintain 
and  improve  the  quality  of  the  course.  During  one  summer,  an  additional 
questionnaire,  the  Computer  Literacy  Attitude  Survey  (described  below),  was 
specifically  developed  to  study  in  depth  the  effects  of  the  course  on  attitudes. 
This  Attitude  Survey  was  administered  at  the  same  time  as  the  regular 
course  surveys. 

Data  Sources 

Seventy-three  students  completed  the  Computer  Literacy  Attitude 
Survey  after  the  first  class.  Sixty-five  students  completed  the  Survey  again 
on  the  next-to-last  class  day.  The  Survey  consisted  of  fifty  Likert-scale  type 
items  derived  from  several  sources  (Abdel,  et  al.,  1986;  Bannon,  Marshall,  & 
Fluegal,  1985;  Ellsworth  &  Bowman;  1982;  Smith,  1987;  and  Violato, 
Marini  &  Hunter,  1989).  The  survey  contained  items  related  to  the  factors  of 
liking  of  computers,  valuing  computers  for  society  and  for  education;  anxiety 
about  using  computers;  confidence  with  regard  to  learning  about  computers; 
and  perceptions  of  gender  appropriateness  of  computers.  Students  were 
asked  to  rate  the  items  from  strongly  agree  to  strongly  disagree. 

Students  also  completed  a  brief  questionnaire  regarding  their 
demographic  characteristics  and  previous  computer  experience. 

Data  Analysis 

First,  several  types  of  descriptive  data  were  collected  and  analyzed. 
Students  were  asked  several  questions  about  their  background,  and 
frequencies  were  calculated  for  these  questions.  Two  questions  each  on  the 
pre-  and  post-course  survey  asked  students  to  rate  their  skill  in  typing  and 
computer  programming,  and  mean  scores  on  these  questions  were  calculated. 

For  the  fifty  Likert-scale  questions  regarding  the  preservice  teachers' 
attitudes  toward  computers,  which  represented  the  primary  focus  of  the 
study,  pre-  and  post-course  mean  scores  were  calculated  .  To  calculate 
statistics,  strongly  agree  was  given  a  value  of  1,  agree  was  given  a  value  of  2, 
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up  t»  strong],  disagree  with  a  velue  of  S.  Thus  on  nositively.worded  items  « 


„„       ^^tennine  the  effects  of  participating  in  the  computer  applications 
a?  thJ ^JS IS SC  t"te8tS        ^  to  comP«™  attitudes  at  the  teginn Te  «d 
oLSw!^         course.  Since  use  of  multiple  t-tests  introduces  Se 
E    I   ^dlne  SPUn!US  s}^^  differences,  a  highly-stringent 
mPBn  W^li    "C  T?S  U8edI  to  <k*«™ine  whether  differenced  were 
meaningfully  significant.  Items  on  which  pre-  and  post-course  differences 
were  significant  at  the  p<.0001  level  were Considered  to  resent  A?mLt 


Results 

A  summnrv  nf  tho  ^Uronr, 

Table  1 


^tw^E  °f  ?e  cniaracteristics  of  the  subjects  is  presented  in 

certified  Ml?h15Udtnt8  Wtre  !,dUuCation  m^ors-  seek*"e  teaching 
™i  n-  Most  h«d  not  completed  their  student  teaching  and  were  not 
SS^  teaching  Students  appeared  to  have  enrolled  in  tL  computer 
course  at  varied  points  in  their  college  career;  about  half  had  comnKed  onlv 
a  few  education  courses,  while  about  half  had  completed  m\re  S  six  * 
SrfM  1     8tuden  *had  had  "ttle  experience  w^th  miSo  omputers  with 

areas,  students'  perceived  that  their  skill  had  impro^edTowfver  the 

ESZSSSZ?  great€r  in  C°mputer  P^BnmS^t^XSLr  to 
Zinr,*  g?d  ^ine  8kil1'  wherea*  their  SS 

sssssK!  risr* could  1,6  improved  con8ide^ 

responJstthVS 

lower  the  mean  score  the  more  positive  fte  .t^^EeS^  J£ 

SSfSSaT  1  (m°f  P0SitiVC)  10  5  0eaSt  ^  ^ondlSlf  of 
iim.  fl  u-  u  3b)  Presente  responses  to  items  worded  negatively   On  these 

Sr-Fefe^ 

As  shown  in  Table  3,  attitudes  toward  computers  as  indicated  bv 

the      (SSiT1?  it6m8  -here  WCre  highly  «&nificant  pre-post  differences  at 
IXESBl&VS^  *• UtUde  ChangCS  reP^Pnte?by  these  Ss 

SfJfff  i"  U»S,ng  COmputers-  that  the  greatest  attitud ^^^poMr  In 
fact,  of  the  twelve  most  significant  attitude  changes,  eight  were  related  to 
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anxiety  and  confidence.  These  items,  noted  in  Table  3,  were  items  #103, 
109  111,  117,  118,  123,  127,  and  129.  For  example,  responses  to  item 
#109  "I  get  a  sinking  feeling  when  I  think  of  trying  to  use  a  computer, 
showed  the  greatest  increase  of  those  to  any  item  (3.45  pre,  4.25  post.  Note 
that  on  negatively-worded  items  a  higher  score  is  more  positive.),  bimilar 
results  occurred  in  response  to  the  other  anxiety/confidence  items:  item 
#103,  "I  feel  confident  with  my  ability  to  learn  about  computers  (2.42  pre, 
1.72  post),  #111,  "Computers  make  me  feel  stupid,"  (2.95  pre,  3.69  post), 
#117,  "I'm  not  the  type  to  do  well  with  computers,"  (3.51  pre,  4.00  post), 
#118',  "I  feel  comfortable  using  computers,"  (2.75  pre,  2.11  post),  #123, 
•Computers  make  me  feel  uncomfortable,"  (3.22  pre,  4.05  post),  #127, 
"Computers  make  me  feel  uneasy  and  confused,"  (3.30  pre,  3.95  post)  and 
#129,  "I  think  using  a  computer  would  be  difficult  for  me,"  (3.51  pre,  3.98 
post). 

Significant  differences  at  the  p<.0001  level  also  occurred  on  three 
items  related  to  liking  computers  (items  #102,  #107,  and  #128).  For 
example,  the  mean  scores  on  one  of  the  liking  items,  #102,  "I  like  using 
computers,"  were  2.27  pre-  and  1.75  post-course.  Finally,  differences  at  this 
highly  significant  level  were  found  on  one  item,  #141,  related  to  valuing 
computers.  The  preservice  teachers  disagreed  more  strongly  at  the  end  of  the 
course  with  the  statement,  "Microcomputers  will  increase  the  amount  of 
stress  and  anxiety  students  experience  in  schools,"  (3.47  pre,  3.64  post). 

Also,  as  noted  on  Table  3,  on  five  additional  items  pre-post 
differences  were  significant  at  the  somewhat  lower  level  of  p<.001.  Again  on 
these  items  the  preservice  teachers  showed  more  positive  attitudes  at  the 
end  of  the  course  with  relation  to  anxiety  towards  (item  #104),  liking  (item 
#121),  and  valuing  computers  (items  #140,  #142,  #147). 

Thus  the  general  pattern  of  significant  differences  on  the  pre-course 
versus  post-course  administration  of  the  Attitude  Survey  indicated  an 
improvement  in  attitudes  toward  computers.  Initially  the  non-significant 
decline  in  scores  on  the  eight  items  mentioned  earlier  was  still  of  some 
concern,  because  five  of  the  items  were  related  to  perceptions  of  computers  as 
a  male  domain,  that  is,  the  scores  seemed  to  indicate  increased  sex- 
stereotyping.  For  example,  scores  on  item  #106,  "Using  a  computer  is  more 
important  for  males  than  females,"  were  4.71  (pre)  and  4.45  (post),  and 
scores  on  item  #130,  "Working  with  computers  is  more  for  males  than 
females,"  decreased  from  4.65  (pre)  to  4.52  (post).  Responses  on  a  few  of  the 
other  items  were  already  very  positive  at  the  beginning  of  the  course  and 
became  only  slightly  less  positive.  For  instance  on  item  #101,  "Knowing  how 
to  use  computers  is  a  worthwhile  and  necessary  skill,"  mean  scores  were 
1.26  (pre)  and  1.42  (post).  Similarly,  in  response  to  item  #112,  "If  a  problem 
is  left  unsolved  in  a  computer  class,  I  would  continue  to  think  about  it 
afterwards,"  students1  mean  scores  were  2.22  (pre)  and  2.42  (post).  As 
mentioned  earlier,  the  decline  in  scores  on  these  eight  items,  although  noted 
on  Table  3,  does  not  appear  to  be  important  in  that  none  of  the  differences 
on  these  items  was  statistically  significant. 


Discussion 


This  study  was  conducted  to  investigate  in-depth  the  effects  of 
participation  in  a  computer  literacy  course  on  the  attitudes  of  preservice 
teachers  towards  computers.  While  many  researchers  have  studied 
computer  attitudes,  few  have  investigated  the  effects  on  attitudes  of  long- 
term  participation  in  a  computer  course.  Of  particular  interest  to  educators 


■  *  j  • 

371 


are  the  effects  of  such  a  course  on  the  attitudes  of  teachers.  This  studv 
extended  previous  work  conducted  with  preservice  teachers. 

In  this  study,  participation  in  the  course  appears  to  have  inmrovpd 
students'  attitudes.  Students1  responses  indicated  that  liked 
computers  more  at  the  end  of  the  course,  had  less  anxiety  about  using 
computers,  had  more  confidence  in  their  ability  to  learn  about  and  use 
computers,  and  generally  valued  computers  more  for  education  and  for 
society  in  general. 

♦  ^  defig?  °f      comPuter  literacy  course  itself  was  studied  in  an 
effort  to  begin  to  determine  why  attitudes  may  have  improved  so  significantlv 
on  so  many  factors,  particularly  anxiety  and  confidence  about  UX  * 

SSSSSh  ?£,,mp0rta,!fe  ^e  fa,ct       *•  course  was  systematically 
deigned,  based  on  sound  principles  of  instructional  development  (cf  Dick  & 

Sr/^1990:  P^etBriffi8.  &  Wager,  1988;  Sullivan  &  Higgins  1?83)  A? 
t  ^T?m%  rf  eachrWent  of  the  course  the  preservice  teachers  were 

tass^st a  set  £  ibjectives  which  cieariy  described  *•  ski»s  ™* 

knowledge  they  would  be  mastering.  Instruction,  frequent  hands-on  oractice 
and  objectives-based  feedback  were  provided.  Tests  and  projects  m^easured 
skills  and  knowledge  the  students  knew  they  were  respond  e  for (Fo *a 
more  complete  description  of  the  course,  see 

Savenye,  Davidson,  and  Smith,  1991).  These  preservice  teachers  in  fact 
throughout  the  course  frequently  mentioned  to  instructors  and  the  professors 
coordinating  the  course  that  they  felt  the  course  was  clear  and  fair  and ThT 
they  were  learning  useful  skills.  They  often  said  the  significant  amounts  of 

^ttJl  PTU1l madC  ft6?  feel  more  comf°rtable  using  computers  now 
and  later  when  they  would  be  teachers. 

«     k    —  m  £  apPears        ^  intense,  systematically -designed  course 
emphasizing  hands-on  computer  skills  teachers  would  use  later  in  ?hefr 
classrooms  may  have  made  the  difference  in  making  aSudes  toward 
computers  more  positive.  While  it  is  difficult  to  manipulate  theTfactors  and 
experiences  ,n  existing  college  courses  which  often  are  required to teach 
certain  skills  by  state  mandate,  future  studies  might  be  conducted^ betrin 
to  parcel  out  which  elements  in  such  a  course  seem  to  be  mos Effective  This 
£  iI!!re>f^eCially  USCfuI  *r  Petitioners  who  are  resZsibKr 

t3rTg  th0rt  *        inService  teacher8'  and  °*er  computer 

trainers  who  need  to  know  how  to  have  the  most  impact  in  the  shortest  time. 

«nn»«r  X  declined  somewhat  on  eight  of  the  fifty  items  does  not 

£££  1* ISSS^STa-  FirSt' thC  differences  wer"  not  statistical^ 
signiiicant.  In  addition,  the  decrease  on  at  east  one  item  was  likelv  to  hnv* 
been  due  to  the  fact  that  the  students  had  just  completed  2,  intSSw 
summer  course  on  computers.  Athough  their  attitudes  were  sbToositive 
students  were  not  as  likely  at  the  end  of  their  perhaps  g^eHng  experience  to 
say,  for  example,  that  they  would  continue  to  think  ibo^tanS Xed 
computer  problem  after  class.  The  declines  on  a  few  of  the oth^r  items  seem 
to  follow  a  pattern  which  appeared  in  the  Savenye,  et  al.,  IMS "taS^JSS 

Soente  Sm^r^r  "f  in*W°?™*  clLroom  uses  of  cZpute^s 
students  seem  to  have  a  less  glowing  overall  opinion  about  the  value  of 
computers  They  were  somewhat  less  likely  to  indicate  that  knowing  how  to 

SSZSS^XT***  and  nece89ary  8kin-  and  »omewhaTmo?e 
2,;  n  ?te  thfu  T  ComtTy  would  *  tetter  off  if  there  were  no 

S«rfte  }a"er  r?P°nse  may  simP,y  be  an  indi"tion  of 
Zucti  computers  at  the  end  of  an  intense  course  of 
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While  the  tendency  of  some  of  these  primarily  female  students  to 
reeard  computers  as  more  of  a  male  domain  after  their  computer  course  than 
before  was  not  significant,  it  may  yet  be  worth  investigating.  While  most  of 
the  students  in  the  course  were  females,  the  two  course  instructors  were 
foreign-bom  males.  As  Bandura  (1986)  has  stated  role  models  can  have  a 
significant  influence  on  behaviors.  In  her  1990  study  on  the  effects  of  role 
models  on  student  attitudes  toward  gender-appropriateness  of  careers, 
Savenye  indicated  that  gender  of  the  role  model  can  influence  perceptions  of 
the  gender-appropriateness  of  activities.  It  appeart,  that  a  worthy  research 
question  would  be  the  effects  of  the  gender  of  the  course  instructor,  and 
possible  interactions  with  gender  of  student  on  student  perceptions  of  how 
appropriate  computer  activities  are  for  men  and  women. 

Another  interesting  investigation  might  be  to  conduct  a  factor 
analysis,  possibly  by  pooling  many  data  sets,  to  determine  how  the  fifty 
items  load  on  factors  derived  from  the  theoretical  and  empirical  literature 
mentioned  in  the  introduction. 


Conclusion 


Many  national  studies  have  called  for  increased  use  of  technology  in 
school  (cf.  U.S.  Congress,  Office  of  Technology  Assessment  1989).  Well- 
trained  teachers  are  the  key  to  effective  use  of  technology.  Snelbecker 
has  suggested  that  teachers  are  the  critical  factor  in  successful 
implementation  of  technology  in  the  schools.  This  study  was  based  on  the 
assumption  that  teacher  success  in  learning  about  technology  is  partially 
dependent  upon  positive  attitudes  toward  technology.  That  such  an 
assumption  is  justified  is  supported  by  the  work  of  such  researchers  as 
Stevens  (1982).  While  this  study  indicates  that  a  systematically-designed 
computer  literacy  course  improves  attitudes  preservice  teachers,  future 
researchers  should  investigate  the  most  powerful  attitudinal  factors  related 
to  computers,  the  instruments  used  to  collect  computer  attitude  data,  the 
effects  of  gender  of  student  and  gender  of  instructor,  and  methods  for  directly 
influencing  attitudes  of  teachers  toward  computers  and  other  technologies. 


(Note-  The  research  reported  in  this  paper  was  partially  supported  by  the 
College  of  Education  at  Arizona  State  University,  and  by  the  Texas  Center 
for  Educational  Technology,  The  University  Research  Institute  of  The 
University  of  Texas  at  Austin,  and  Project  Quest  of  The  University  of  Texas 
at  Austin.  The  author  would  like  to  thank  Dr.  Gayle  Davidson  for  her  efforts 
in  collaborating  to  collect  the  data,  Dr.  Patricia  Smith  for  her  work  in 
developing  the  original  computer  literacy  course,  the  instructors  and  students 
in  the  area  of  Instructional  Technology,  and  several  anonymous  reviewers  for 
their  helpful  comments.) 
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Table  1 

CharaCtftriSfks  nf  the  Ptrsrrvicc  Tether*  m  the  Cnmpnt^r  Attitude  Study 


Characteristics 


1. 

Major? 

Education 

Other 

45 

15 

(66.2%) 

(33.8) 

2. 

Seeking  certification'' 

ICS 

XT  -~ 

No 

58 

1 5 

(79.5%) 

(20.5%) 

3. 

Completed  student  teaching? 

Yes 

No 

18 

54 

(25.0%) 

(75.0%) 

4. 

Currently  teaching? 

Yes 

No 

13 

60 

5. 

Number  of  previous  courses 

(17.8%) 

(82.2%) 

in  education? 

0  1-2 

3-4 

11  23 

11 

5-6 
7 


>6  courses 
21 


6.     Previous  experience 

operating  microcomputers? 


Own  or  have  easy  access 
to  a  microcomputer? 


TViken  any  formal  or  informal 
courses  in  use  of  computers? 


(15.1%)  (31.5%)  (15.1%)  (9.6%)  (28.8%) 

0      few  days  1-6  mos.  7  mos  >1  yr. 
to  1  mo. 

15         16  9         8  25 
(20.5%)  (21.9%)  (12.3%)  (11.0%)  (34.2%) 

Yes  No  Missing 

29  43  l 

(39.7%)  (58.9%)  (1.4%) 


Yes  No 
31 

(42.5%) 


42 

(57.5%) 


Table  2 

Pre-  and  Post-Course  M»*n  Self-R«rings  q£  Skjlk 


Competency 


PRE  Mean     POST  Mean 


A. 
B. 


How  would  you  rate  your  typing  skill?  2.58 
How  would  you  rate  your  computer 
programming  skill  (in  any  language)?    1 .34 


2.91 
2.82 


Njye,  Scores  range  from  1  (nonexistent)  through  2  (poor),  and  3  (good)  to  4  (excellent). 
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Table  3a 

ad  Eas^cam  „  I  leap  Semes  nn  Camniugr  At»tudes 


Items  Worded  Positively 
Mean 


PRE  Mean  POST 


I  iiflpg  of  rnmruters 

102.    I  like  using  computers. 

107     I  like  using  computers  in  my  school  work. 

108.    I  wish  I  could  use  computers  more 
frequently  at  the  university. 

1 1 0.    Once  I  start  to  work  with  the  computer, 
I  would  find  it  hard  to  stop. 

112     If  a  problem  is  left  unsolved  in  a  computer  class, 
I  would  continue  to  think  about  it  afterwards. 

114.    A  job  using  computers  would  be  very  interesting. 

116.    I  look  forwand  to  using  the  computers  at  the  university. 

121     When  there  is  a  problem  with  a  computer 
program  I  can't  immediately  solve,  I  would 
stick  with  it  until  I  have  the  answer. 

128.    1  think  working  with  computers  would  be  both 
enjoyable  and  stimulating. 

1 49.    Someday  I  will  have  a  computer  in  my  home. 

Anxiety.  CQLLacii  rrf  It)  AN"ir  Camcmsis 
Value  of  rnmpntnr fnr  Society 

101 .    Knowing  how  to  use  computers  is 
a  worthwhile  and  necessary  skill. 

1 32 .    1  will  probably  need  to  know  how  to  use  a 
computer  when  I  leave  school. 

\/«i.T  nf  q  pmpiue.rs  ffr  Education 

1 05 .    I  will  use  my  knowledge  of  computers 
in  many  ways  as  a  teacher. 

1 22     Learning  about  computers  is  a  worthwhile  and 
necessary  subject  for  all  prospective  teachers. 


2.49 

2.98 

2.22 
2.74 
2.41 


1.26 
1.88 

2.27 
1.71 


2.27  1-75 


2.38  1-80 


** 


** 


2.20 

2.58 

2.42 
2.55 
2.17 


2.62         2.18  * 

2.33  2.23  ** 
1.69  1.55 


1.42 
1.68 

2.02 
1.71 
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Table  3a,  cont. 

Pre-  and  Post-Course  Mean  Scores  on  CantmsL  Arrimrir 5 


Items  Worded  Positively  ,  cont. 
Value  Of  Computers  fnr  Fjjucarinn  cnn| 


PRE  Mean    POST  Mean 


124. 


It  is  important  to  know  how  to  use  computers 
in  order  to  get  a  teaching  position. 

Supplying  every  student  with  a  microcomputer 
is  a  worthy  educational  objective. 

Teachers  should  demand  that  they  be  taught  how 
to  use  microcomputers  in  their  classrooms. 

Microcomputers  will  require  students  to 
become  active  learners. 

USC  micr&r?mputers  in  school  instruction,  our 
students  will  grow  up  illiterate  and  deprived  of  a  basic  skill. 

If  my  school  district  had  the  money,  I  would  insist  that 
they  buy  microcomputers  in  most  every  school  subject. 

Computers  can  improve  learning  of  higher-order  skills. 
Computers  will  improve  education. 
Confidents  About  T  ^rnjng  Aho.it  Comnmnrn 
103.     I  feel  confident  about  my  ability  to  learn  about  computers. 

1 25.     I  know  that  if  I  work  hard  to  learn  about 
computers,  I  can  do  well. 

Confidence,  About  Tking  Cnmpmrrn 

1 18.    I  feel  comfortable  using  computers. 

131.    I  am  able  to  do  as  well  working  with 

computers  as  most  of  my  fellow  students. 


134. 

135. 

136. 

139. 

140. 

145. 
150. 


2.77 

2.38 

2.54 

2.39 

3.08 

2.81 
2.21 
1.80 

2.42 
1.64 

2.75 
2.33 


2.66 

2.23 

2.24 

2.17 

3.00 

2.38  * 

2.13 

1.75 

1.72  ** 
1.60 

2.11  ** 
1.97 

1.38  -- 


Perceptions  AppUt  fonrirr- Appropriate^  nf  ^mmrr  I ' t 
1 26.    Females  can  do  as  well  as  males  in  learning  about  computers.   1  36 

pre/post  differences  significant  at  the  p<.001  level 
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Table  3b 

Pre-  and  Pofjt.fYmrse  Mean  Scores  on  Cnmnntcr  Attitudes 

Items  Worded  Negatively  PRE  Mean  POST 

Mean 


1  .i|qnc  of  Computers 

1 19.    Working  with  computers  is  boring. 

Anxiety  fQr  l*r\c  of  irt  About  Computers 

104.    Working  with  a  computer  would  make  me  nervous. 

109.    I  get  a  sinking  feeling  when  I  think  of 
trying  to  use  a  computer. 


3.68  4.03 

3.48  3.98  * 

3.45  4.25  ** 

2.95  3.69  ** 


111.  Computers  make  me  feel  stupid. 

117.  I'm  not  the  type  to  do  well  with  computers. 

123.  Computers  make  me  feel  uncomfortable. 

1 27.  Compu^; s  make  me  feel  uneasy  and  confused. 

129.  I  think  using  a  computer  would  be  difficult  for  me.  3.5 1  3.98 
Value  of  Compptftrs  far  Society 

1 15.  I  don't  expect  to  use  computers  when  I  get  out  of  school.        4.03  4.23 

1 33.  Computers  are  gaining  too  much  control  over  people's  lives.     3.35  3.66 


3.51         4.00  ** 

3.22         4.05  ** 

3.30        3.95  ** 
** 


879 


O  w.  lJ 


:RIC 


Table  3b,  cont. 

Pre-  and  Post-Course  Mr.an  Scones  qd  r^mer  AttimH^ 

Items  Worded  Negatively,  cont  PRE  Mean   POST  Mean 


3.73 


Value  Of  Compute  %  Education,  ^f>nt, 

1 37 .  Microcomputer  instruction  will  deny  students 
the  opportunity  to  reason  with  others.  3 46 

138.  Using  microcomputers  as  a  teaching  tool  puts  too  much 

additional  work  on  already  overbunlened  teachers.  3.6O        3  73 

141 .  Microcomputers  will  increase  the  amount  of  stress 

and  anxiety  students  experience  in  schools.  3.47         3  M 

1 42.  Microcomputers  will  decrease  the  amount  of 

teacher-pupil  interaction  in  schools.  3  j  5         3  5  g 

143.  Microcomputers  will  isolate  students  from  one  another.  3.33  3.59 

1 44.  I  object  to  all  the  attention  being  given  to  computer 
technology  because  it  detracts  from  the  real  problems 
now  faced  by  teachers.  3  50 

146.  Computers  will  displace  teachers.  411 

147.  Computers  will  dehumanize  teaching.  379  493 

148.  Our  country  would  be  better  off  if  there  were  no  computers.  4.48  4.39 
Confidence  About  t  raping  Ahom  Computers 

Confidence  Ahnnt  I^pp  C.nm^m 

Perceptions  About  GsndcLABBBaaiaiBDess  "f  Cswbwl  lias 

106.    Using  a  computer  is  more  important  for  males  than  females.    4.71  4.45 

1 1 3.    More  men  than  women  have  the  ability  to  become 

computer  scientists.  A  ,0 

4.62  4.58 

1 20.  Using  computers  is  more  enjoyable  for  males  than  females.  4.60 
1 30.    Working  with  computers  is  more  for  males  than  females.  4.65 


3.69 
4.33 


4.56 
4.52 


-  pre/post  differences  significant  at  the  p<.00l  level 
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Classifying  Interaction  for 
Emerging  Technologies  and  Implications 
for  Learner  Control 


Abstract 


Snfrn^nHiT11^8  °  ^^ation  scheme  for  multimedia  interaction  based  on  the  degree  oJ 
control  and  type  of  cognitive  engagement  experienced  by  learners  in  orescriotive  dWoSSr 

S^metiC^ependent  learnin«  environments.  ReLi^p^XaSm^^eSo; 
interaction,  and  their  associated  funtions  and  transactions  are  discussed  ThTn^er  also 

HnP^fnSiinnClplSS  f°r  desi8n,"g  interactive  multimedia  instruction  whSh emer? from  this 
classification  and  current  research  on  learner  controL  emerge  irom  tnis 


Genesis  of  a  Taxonomy  of  Interaction 

SiS^?,!^^™1  **™*rgte  t^^nologies  grew  out  of  the  interactive 
cmflri  on^t^^  "       ♦     5     2"  term  levels  t0  des"-ihe  hardware-dependent  ways  learners 
could  operate  equipment  and  software.  Commonly  accepted  labels  for  levels  of  interactiviri 
were,  and  remain.  Level  I,  II.  III.  and  sometimes  IV.  depending  on  vour referee  oESZ  vSh 
Kafzand !  Keet  1990;  Ka*  1992;  Schwartz  1987;  Schwler.  1987?.  LevcTl ^SSSSS^SR 
f2nf *e  leaTer  With  a  remote  contr°l  "n^  to  access  frames  or  Sments  on  the  discs 
Level  II  interactivity  featured  control  programs  recorded,  on  videodiscs  which coul FoJv  bf 

1Z™£,  a  ZZ,  d!d  by  an  external  computer  connected  to  a  player.  Level  IV  usuallv 
introduced  sophisticated  interfaces,  such  as  touch  screens,  into  the  s^tem.  * 

These  levels  say  little  about  the  quality  of  interaction  engaged  by  the  learner  (see  Figure  1) 

ht ^£2SSL?alhar  bCen  °r  c°ntinues  to  be  for  interactive  video,  for  environments  created 
by  emerging  technologies  it  is  more  productive  to  characterize  interaction  acco3?J!l  tn  S 
sophistication  and  quality  of  ir.teraction  available  to  a  l^Ta  p^Sp^^ 
Figure  1.    Interactive  features  of  level  I.  II  and  III  videodiscs. 
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A  Taxonomy  of  Interaction  for  Emerging  Technologies 

Multimedia-based  independent  learning  happens  in  several  contexts,  depending  upon  the 
nature  of  the  learning  task,  the  abilities  and  preferences  of  the  learner,  and  the  external 
requirements  of  the  learning  context  Learning  may  occur  in  prescriptive,  democratic  or 
cybernetic  environments,  each  oi  which  may  be  appropriate  or  inappropriate  to  accomplish 
defined  learning  tasks  (Romiszowski,  1986). 

Instructional  designers  acknowledge  the  role  played  by  prescriptive  types  of  instruction; 
indeed,  prescriptive  instruc  on  has  dominated  the  attention  of  instructional  design  for 
decades.  Some  types  of  learning,  say  performing  double-ledger  accounting,  maybe 
appropriately  addressed  in  a  confined,  externally  defined  and  structured,  highly 
proceduralized  fashion.  An  instructional  designer  can  develop  effective,  reliable  instructional 
materials  to  address  this  problem. 

But  emerging  technologies  require  us  to  look  at  independent  learning  systems  in  a  new  way-as 
environments  which  promote  the  learner's  role  in  regulating  learning.  Emerging  technologies 
focus  on  an  ability  to  manage,  deliver  and  control  a  wide  range  of  educational  activities 
(Hannalin,  1992).  Instructional  designers  must  look  beyond  the  attributes  and  differences  ol 
individual  media  components,  and  instead  extend  attributes  across  developing  technologies. 
To  fully  exploit  the  capabilities  oi  more  powerful  instructional  technologies,  designers  must 
also  reexamine  the  assumptions,  models  and  strategies  we  employ  in  instructional  design 
(Cognition  and  Technology  Group  at  Vanderbilt  1992;  Jonassen,  1991;  Osman  and  Hannafia 
1992;  Rieber,  1992;  Schott,  1992;  Spector,  Muraida  and  Marlino,  1992;  Tennyson,  Elmore  and 
Snyder,  1992). 

Emerging  technologies  offer  an  expanding  range  of  interactive  possibilities  which  are 
remarkably  consistent  regardless  of  the  platform  used  to  deliver  the  instruction.  Because  a 
computer  acts  as  the  heart  of  a  multimedia  learning  system,  and  because  most  multimedia 
computer  systems  have  similar  devices  for  communicating  (e.g„  keyboard,  mouse,  touch 
screea  voice  synthesis),  the  quality  of  interaction  is  more  the  product  of  the  way  instruction  is 
designed,  and  less  the  result  of  the  system  on  which  it  is  delivered  In  order  to  describe  a 
taxonomy  of  interaction  for  multimedia  instruction,  this  paper  describes  three  learning 
environments  within  which  interactive  multimedia  functioa  suggests  three  levels  of 
interaction  commensurate  with  these  environments,  examines  functions  played  by 
interaction  within  these  levels  and  enumerates  several  types  of  overt  transactions  available  at 
each  functional  level  of  interaction  (Figure  2). 

Figure  2.  A  taxonomy  of  interaction  for  multimedia  instruction. 
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Multimedia  Learning  Environments 

Instruction  is  shaped  by  the  instructional  designer's  knowledge  of  the  learning  task,  learner 

m  SJJTJ ?S  orl=ntat,on  of  the  learning  materials.  These  factors,  among  others,  conspire 
to  define  a  multimedia  environment  Romizowski  ( 1986)  used  me  terms  prescriptive 

^SSS^^S^S  SChCmata  °f  SyStems  for  ^ividualizing  insiruction: 

systems  which  may  also  be  considered  environments  in  multimedia  instruction. 

mSSSSZJS^ U°P  SPecif,fs  what  the  learner  is  to  leara  Instruction  is  based  on  specific 
objectives  and  the  instructional  system  is  used  as  a  primary  delivery  medium.  In  mam,  if  ™t 
most,  cases  the  instructional  content  and  bounciariesVSSiSi ^dSdSMS^ 
ms^ructional  designer,  and  the  learner's  role  is  to  receive  and  master  the  given  content  A 

SSSS^A^^TSSu^  ^J^I  ^simulations  (e.g.  Alessi  and  Trollip.  19?5; 
Mannann  and  Peck.  1988;  Heinich.  Molenda,  and  Russell.  1989;  Ron  .szovski  1386)  Rerentlv 
tartructional  design  theory  has  offered  models  for  dealing  with  WgheJoS 
more  complex  cognitive  skills  with  prescriptive  design  models  based  on  cogn  tTvTthfor?  f  e  * 
Mernenboer.  Jeisma.&  Paas.  1992,-Tennyson,  Elmore.  &  Snyder  1992)  i^u*  JShSTSS- 
do  not  offer  a  complete  theory  of  instruction^  design  liSSS^^^^toS^ 
potentially  powerful  solutions  to  specific  design  challenges  (Schott.  1992) I  V 
Democratic  environments  turn  over  control  of  instruction  to  the  user.  Unlike  orescriotive 

ontteTaSr  t^L^^T  d°       imP°Se  hlgWy  ^cturXarnSg \ Z  gies 
what  is  lSrSd 5  f«  ^  JC  e"v^onmcnts  emphasize  the  learner's  role  in  defining8 
Am~  ^^r  lcamed  and  the  sequence  in  which  it  is  learned.  The  most  apparent 

difference  between  democratic  and  prescriptive  environments  is  the  level  of  learner SEE1* 
and  they  do  not  always  operate  in  isolation  from  one  another  DeZcZtc  envKcnTs  mav 
be  used  to  support  prescriptive  instruction,  acting  as  a  supplementary  resource  to  STe DnmT™ 

SS^rf'  8  1Crer  f°ll0Wing  8  -^nrtnSionl  P^gram  on  a  compa^Sl 
British  and  American  forms  of  government  (prescriptive)  might  choose  to  explore  a  leamim? 
resource  on  the  Canadian  House  of  Commons  to  elaborate  inforrnation ,  for  an ^assiPnm^t  g 
(democratic).  For  other  democratically  oriented  learning,  resVuTces rand JE  ™  fiZS 

to^&ZS^^Sr- a?d  me  learner  makes  virtuallv  ^^-ioT£S5  how 

and™n^t^  ligation,  motivation 

?»bC.mCtiC.  environments  emphasize  a  complete,  multi-faceted  system  in  which  the  learner 
IZmfte^r^'^  ^helically.  Intelligent  interact^  multimed^nased on  expert 
ow.f^ai  he<irisnc  designs,  and  virtual  reality  can  provide  rich,  dynamic  and  realistic 

P  ma^co^  - 

J   ?V         ,     learning.  but  the  system  continually  adapts  to  learner  activitv  anri  m«« 
-  ven  adapt  in  novel  ways  based  on  heuristic  interpretations  of  learner  acUons The  LrntT* 

st&ir1" lcve,s- ac,uauy        *•  «■«*"  ^™8n  dCnc,s,„„s 
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specific  content  to  be  learned  and  generalize  to  other  settings,  rather  than  focus  solely  on 
specific  content  and  skills  to  be  learned. 

Levels  of  Interaction 

The  different  multimedia  environments  will  emphasize  different  levels  of  interaction.  Such 
interaction  can  be  characterized  as  reactive,  proactive  or  mutual  depending  upon  the  level  01 
engagement  experienced  by  the  learner. 

In  a  reactive  interaction  a  learner  responds  to  stimuli  presented  to  the  learner  by  the  Pr°gra™- 
for  example  by  making  a  selection  from  a  menu  (Lucas,  i992,  Thompson  and  Jorgensen,  1989). 
Such  approaches  are  typified  by  tutorial  designs  wherein  the  learner  and  computer  are 
engaged  in  a  preordained  discussion  which  is  initiated  by  the  program,  not  the  learner. 

Bv  contrast,  proactive  interaction  requires  the  learner  to  initiate  action  or  dialog  with  the 
program.  Proactive  interaction  promotes  generative  activity:  that  is.  the  learner  goes  beyond 
selecting  or  responding  to  existing  structures  and  begins  to  generate  unique  approaches  and 
constructions  other  than  those  provided  in  instructional  materials.  An  example  of  this  is 
when  a  learner  uses  key  word  searching  of  a  hypermedia  database,  organizes  resultant 
information  to  address  a  self-generated  question. 

The  highest  level  of  interaction,  mutual  interaction  is  characterized  by  artificial  intelligence 
or  virtual  reality  designs.  In  such  programs,  the  learner  and  system  are  mutually  adaptive. 
Sometimes  this  is  referred  to  as  recursive  interaction.  Recursion  is  based  on  the  mathematical 
notion  of  indefinite  repetition  and  in  multimedia,  it  suggests  a  conversation  which  can 
continue  indefinitely.  This  is  a  useful  distinction,  but  it  falls  short  of  the  potential  capabilities 
of  multimedia  systems  in  the  future.  Because  multimedia  systems  may  ultimately  be  capable  of 
cybernetic  convcrsation-actually  learning  from  and  adapting  to  conversation  with  a  learner- 
thc  term  mutual  interaction  is  used  here.  At  a  less  sophisticated  level  mutual  interaction  can 
be  used  to  describe  the  appearance  or  trappings  of  meaningful  conversation  Mutual 
interactivity  is  still  in  its  infancy,  but  the  area  is  attracting  a  great  deal  of  research  and 
development  interest. 

The  three  categories  of  interaction  do  not  exist  as  "pure"  categories  in  most  instructional 
software-interactive  multimedia  programs  often  incorporate  a  combination  of  reactive  and 
proactive  approaches  (although  very  few  are  sophisticated  enough  to  mcorporate  mutual 
approaches).  But  the  levels  are  hierarchical  in  that  one  subsumes  the  other.  In  other  words, 
mutual  interactions  contain  proactive  elements,  and  proactive  interactions  contain  reactive 
elements  For  example,  when  learners  generate  new  questions  and  approaches  (proactive)  they 
can  in  turn,  be  used  by  the  system  to  formulate  new  conversation  (mutual).  Similarly,  when 
learners  generate  their  own  strategies  (proactive)  they  are  responding  to  existing  stimuli  at  a 
sophisticated  level  (reactive). 

Functions  of  Interaction 

Hannafin  (1989)  identified  five  functions  interaction  can  serve  in  independent  learning 
materials:  confirmation,  pacing,  inquiry,  navigation  and  elaboration.  Confirmation  verifies 
whether  intended  learning  has  occurred  (c.g..  learners  responding  to  questions  during 
instruction  can  measure  performance).  Pacing  gives  control  of  the  timing  of  instruction  to  the 
learner  (e.g..  the  learners  selecting  an  abbreviated  or  elaborated  version  of  instructional 
content)  Navigation  determines  the  amount  of  freedom  and  ease  of  access  learners  have  to 
instructional  components  (e.g.  learners  choosing  segments  from  a  menu).  Inquiry  allows 
learners  to  ask  questions  or  construct  individual  pathways  through  instruction  (e.g.  learners 
searching  supplementary  material).  Elaboration  allows  learners  to  move  from  known  to 
unknown  information  or  expand  what  is  already  known. 

Each  function  is  expressed  differently  during  instruction,  depending  upon  the  level  of 
interaction  For  example,  reactive  navigation  is  typified  by  menus  or  prescribed  branching 
options  presented  to  learners.  Proactive  navigation,  by  contrast,  would  permit  the  learner  to 
initiate  searches  or  participate  in  open-architecture  movement  throughout  materiaL  Mutual 
navigation  might  happen  when  a  program  anticipates  navigation  routes  of  the  learner  based 
on  previous  movement,  and  advises  the  learner  about  the  nature  of  choices  made.  In  mutual 
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Transactions  During  Interaction 

employed  whenever  overt  transactions  are^iavaiS  £?Sf  i      covf  ?  transactl°ns  can  be 
not  preclude  the  use  of  another  unavailable  to  the  learner.  Also,  the  use  of  one  does 

cannot  be  easily  adapted  to  highe ^vlte^nn  it     transactions.  Several  transactions 

interactions  is  confined  i  'a  Z» f*      ™f  '   °S  examPle- the  ^nge  of  possible 

such  as  the  mouJ ?S  l£^S2S^ejffiS^S  2S?CSafi  ^  8  Devices 

as  on-line^note  taking  (i^vJT^ttoSr^'  ^  S™""^  ^"actions  «* 

™^  Actions  can  also  be  used 

compose  a  new  ■o^^tfSffl^SfSe^-^  *™  bl!1UBed  *  a  ,eamer  to 
to  have  learners  mini?  a  score  S«db??^S?™w ^  keyboard  synthesizer  can  be  used 
transactions  conform  t "the °hiera?chJ %t  tEES£  (*a*?e  Inter«ction).  In  this  way, 
higher  levels  of  int^l^cS^S^^Vl^y-  Tfransacti°nal  events  available  for 
not  reciprocal.  P  d  t0  loWer  lcveIs  of  interaction.  but  the  relationship  is 

interactive  strateg ie empiric Hn taSS^SS,1'^01  exhaustive'  but  «  illustrates  some 
notion  that  as  interaction  reaches  o n?Se f    *  Pro2rams-  The  illustrate  the 

transactions  are  required  g  18  °f  en#«eraent  with  learners,  generative 
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Figure  4.  Some  examples  of  transactions  available  to  serve  different  functions  and  levels  of  interaction.2 
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-think  about  tote  image" ).  wouia  oe  restricted  to  covert  responses  to  stimuli  (eg, 

SAfft^JSE t0  *  "* proactlve- only  °vert — *-* 

'ompix  SSff  imP"eS  ,nformation  ^tween  user  and  system,  therefore  requiring 
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Implications  of  the  Taxonomy  ior  Instructional  Design 

E^T££!2SS  ma?£e?..Tn££>«r  or  thin*,.  Learner,  may  be  granted  or  may 
require  control  over 

•  Content  of  instruction. 
.  Context  for  learning. 

•  Presentation  method  of  the  content 

•  Provision  of  optional  content. 

•  Sequence  of  material  to  be  learned 

•  Amount  of  practice. 
.   Level  of  difficulty. 

•  Level  of  advisement. 

The  taxonomy  is  mean,  to  be  K  o,  fmeracttn  are  tended 

»ZT< IS"  oontnl,  o.cr  the  content  it*  Pre"n«i„a  jgu^j^kjd  <* 

mMWMmm^ 

Sme  Smo^rof  learner  control  is  shared  at  a  mutual  level  of  interaction. 

».?a™k aaSTfo  perfom "£cc"css£ny"n  less-stmctured  learning  en.ironm.nta  Osman  and 
Hanl^nn  1^2)  point  out that  significant  variables  in  me  acquisition _jndu» e  o 

SefCg^ 
strategies 

Decisions  about  control  form  part  ol  the  art  of  instructional  design.  One  should  not  assume 
mat ^proTctTve  and  mutual  forms  of  interaction  do  not  impose  external  e^ments  o garner 
control.  On  the  contrary,  considerable  control  of  the  le?mcrthc™ 

inst  ructional  designer  in  subtle  and  passive  forms,  such  as  the  design  of  the  access  structure 
^^^X^StFuc  examPleP  confusing  or  obscure  icons  may  discourage  learners 
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frustrate  learners  fTorn  excrXg  S  contSlTn  other  S£  ^T?  °f  *  de8lgn  wlth  ««* 
to  learners,  It  must  be  ft,/™  by  learners and  d'J^l  5 f  »      dR"  C?ntro1  mu8t  not  onlV  be  tf'wi 
decision  to  take  control.  ,e4,rnerfi-  «na  design  factors  may  inhibit  or  encourage  their 

SSSf  COnlr01  *»  """P"**  ^ing  designs  would  be 

General  Conclusion*  About  Control 

„         .  *  preferences,  strategies  and  processes  used  by  the  learner 

SSfBaft"^  —  -V  ^crease  ' 

'   &ir0?c?^  t0  016  •  so  also  is  the  external 

may  increase  t^^SS^^S^^S^^  «*»«*■««  — 

'  -*» t  how  much  practice 

hand,  metacognitive  strategies  can  be  alSed "  (R^\ 1984)"  °n  tne  other 
learner  make  more  producuve  da*£^%£^^  he* 

Control  Issues  Related  to  Learner  Characteristics 

'  XSS  SJSSi^SRS^  -knowledgeable  about  an  area  ol 
Hannafin  and  Colamaio  1987)  8         tearncr  COntro1  (Bo"ooK  1991;  Gay.  1986; 

'  ^^I^Sr^'S  S,tmCtUre>  interaction.  «nd  feedback  ro  perform 

1990;  KiS SSSaS^ ^^"SSS,  ^  "^l^^ham-Wheat.S"11 
.  Thr  eff^-  r.  Schloss.  Wisniewski,  and  Cartwright,  1988). 

>eamer  characteristics  as 
Okey,  1990).  *  subject  matter,  and  locus  of  control  (Santiago  and 

Control  Issues  Related  to  Program  Variables 

'  ^cing^  to  unstructured  learner  control  for 

challenge  (Arnone  and  GribowS?   99 ?• hT°1 ^-on-task.  and  stimulating  self- 
1991;  Milheim  and  A.beH,^^  and  flurmar, 

O'Dell,  1990).  DCl1  1988)  dnd  context  conditions  (Ross,  Morrison,  and 

^quene,  and  -JS2a!5StfSSUE^  SSg"  — "~  such  as 
Control  Issues  Related  to  Practice 

'  cognitive  strategies,  such  as 

transfer  (Osman  ft  Hannafla  lSjl&ft  iwS?     C°mP     1Carni"8  and 
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strategies  to  be  successful  (Hooper.  1992). 

Q   _i      um.m  ;«^„Hr  nractice  with  strategies  for  learning,  not  just  practice  with 

practiced  (Osman  &  Hannafin.  1992). 
.    Practice  should  be  available  » the ^'Zi^iSS^SuS  wTu'S  im™ed  Sen* 

(Salisbury.  Richards.  &  Klein.  1985). 
.    Practice  during  instruction  should  be  varied 

.    As  facility  and  familiarity  with  the  ^^^S^^^^X^ 
practice.  In  prescriptive  envxroniMnt*  toe  difflc^  1^1  w °*<^m  learner 

in  the  hands  of  the  learner. 
.    Practice  events  should  require  learners  to  use  information  and  discover  and  derive  new 
relationships  in  information. 

These  suggestions,  however  inviting,  should  be  W™^™*^1™™^^ 
oSuS^ccS  poStton  in  multimedia  research  during  this  decade. 

V 

Summary 

The  Cassation  ot interaction  *<-'*--*£*^ 

and  developmental  As  '.^huct..^  desgn  meory  ^offered  herein  win  likely  evolve, 

by  interaction  during  instruction. 
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Teaching  Collaborative  Problem  Solving  Using  Computor-Modiatod  Communications 

Faculties  in  schools  of  business  nro  continually  challenged  to  find  ™Ipv»m 
and  effective  way*  of  leaching  management  theory  and  praS  UHneir 

MilZ 8  grUdUate  buBineHS  m«»T*  <radi,Tona  Intent 

Z££  °h  manf«emen1t  Pnnciplea,  three  additional  area*  of  ins. ruction  need  to  be 

Z^£ffit.l!Tre  th°SV;Uient8  for  thc  P«*— oorf  role"  the y  wi  I 
f I™         ^       !       •  busme88  w»rl"   P'rst,  students  need  to  become  acliVe 

^^tiSS!;  thev  Have  ,earned  Second- sti,denui  stsl 

solving  real  world  problems  presented  in  a  comp  ex  framework  And  third 

The  business  case  studies  that  are  commonly  assigned  as  erouo  Dmiero,  in 

foratofror6"16"1  C°Ur8eS  d°  8erVC  to  meet  these  eduStiona.ZIP?he  yp"  al 
format  of  such  assignments  requires  the  students  to  analyze  the  strengths 
weaknesses,  opportunities,  and  threats  of  a  company,  then  kCnV^XSc  issues 
and  formulate  recommendations  for  that  company.  Students  mSCSSiT^ 
required  to  research  the  information  on  the  history  of  the  company  aZ  Kdustry 
Produ°e7m7anTr:rd  P,U,P0Se'  eX,°rnal  envi— '  •  ^rnal  env^leT' 

CSrT^lt  3TI*B' compfitlt,ve  market- and  strat«gies  U8ed  by the 

.«,■      Th®re.are  aome  Problems  associated  with  these  group  case  study 
assignment*,  however  Assigning  grades  to  student*  can  be  problematic  because  the 
nstructor  cannot  easdy  ascertain  the  contribution  level  of  each  student  to£ roper 
that  was  handed  ,n.  Even  when  the  instructor  has  students  iden.tfy the ir 
contributes  to  the  report,  grading  inequities  are  difficult  to  avoT  Most 

ZIh  Ct0:rd°PVthe  P,°,iCy  °f  giving  a11  member«  of  the  group  ™  8a™  grade 
based  on  the  qual.ty  of  their  joint  effort.  Better  teamwork  may  result  but X 
problem  of  grade  inequities  is  not  resolved  by  this  approach 

„™ki  Ja?'.,t.he  Hsmg,e  grade"  P°licv  mav  in  and  of  itself  intensify  grade  ineouilv 
S  y,[a'bng  10  Pr°Vide  8ny  con«en««nees  for  an  unequal  distriK  ofwork 

oad  wuhm  the  group.  Some  students  will  become  "free  riders."  taking advance  of 

iTentaThn  68  TlT ^  »«*  *  g00d  grade  °"  the  ^nmel  Other  " 
students  who  are  h.ghly  motivated  by  grade  and  who  lack  confidence  in  their 

assured  often  dommate  group  discussions;  students  who  dislike  «fflS,he 
bePSenn;y  *  *  ^  °'  Wh°      — f-table  defending  th^ZtX 
Knft*  SyS'em  £  m'crocomputers  interconnected  with  electronic  meeting  system 

e!l2?ZS        m  th'S  ^SearCh  S,udy  as  8  P°ssib,e  "»«>"-  of  overcom  ng  he 
Si  1h    PrartlC,pat,on  dlsParity  l>r«blems  associated  with  group  work  so  sions 
conducted  m  face-fo-face  meetings.  Studies  published  by  users  of  SroupSys"  ems  an 
e^ctromc  meetmg system  software  package  developed  a.  (he  UnjveSTXSna 
reported  that  computcr-mediatcd  communications  lend  to  redu^Xhnc ionT 
personal  mteract.ons  such  as  domination  by  some  group  members  and pa Soation 
avoidance  by  others  as  well  as  increasing  productivity  for  the  group  aVa  whole 
Another  advantage  for  the  instructor  is  the  feature  in  GroupSystems  tha\7ro*dos 
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in  collaborative  problem-solving  activities.  The  expectation  of  the  authors  was  that 
Lrom* Sen*  collaborating  on  case  study  analysis  assignments  would  be  more 
prZctte  and  would  participate  more  uniformly  when  communicating  with  each 
other  through  networked  computers  than  in  a  face-to-face  meeting. 

Method 

^^I'wentv  undergraduate  business  students  at  the  University  of  Northern 

Colorado  were  used  as  subjects  for  this  study.  Five  females  and  fifteen  males 

n  m, rt Tthe  X  All  were  enrolled  in  a  senior-level  "Strategic  Management  and 

tSSmPda*-  class  taught  as  a  capstone  course  or  business  majors.  The  class 

was  dSed  into  five  groups  of  four  students  each  by  arbitrary  assigning  a 

marketing  major,  an  accounting  major,  a  management  major,  and  one  of  the 

XaLng  students  to  each  group.  Each  of  the  five  groups  was 

seated  to  be  either  an  experimental  group  or  a  control  group.  Three  groups  were 

designated  as  experimental  groups  and  two  as  control  groups. 

^^ase  studies  on  four  different  corporations  were  prepared  for  use  as 
collaborative  problem-solving  assignments.  Kach  case  study  contained  information 
about  the  historv  of  the  company  and  its  industry,  the  company's  mission  and 
purpose,  external  environment,  internal  environment,  product  mix  and 
tat  eristics,  competitive  market,  and  strategies  used  by  the  company  to  meet  its 
goa     Group  members  »ero  given  copies  of  the  case  study  pnor  to  the  group  s 
scheduled  meeting  to  produce  a  written  analysis  of  the  strengths,  weaknesses, 
S,Wrtunities.  thrLts.  strategic  issues,  and  recommendations  for  that  company. 
Fa  o-Ui-face  group  meetings  were  conducted  in  classrooms  with  the  four  students 
sea  ed  around  a  threc-foot  by  five-foot  table.  The  group  meetings  using  computer- 
mediated  communications  were  held  in  the  Decision  Support  CJnter  a  -eting 
room  with  twentv-one  networked  PCs  running  under  Groupfeystems  V,  an  electronic 
meeting  system  software  package  marketed  by  Vent  ana  Corporation  in  lucson 
Arizona  The  "Topic  Commen.er"  tool  in  the  package  was  used,  allowing  students  to 
enter  their  comments  for  each  of  the  six  analysis  areas  (strengths^weaknesses. 
opportunities,  threats,  strategic  issues,  and  recommendations)  and  to  view  the 
comments  of  their  teammates. 

Procedure^       ^  assignments  used  in  this  research  were  included  as  part  of 
the  class  requirements  and  made  up  a  substant  ial  part  of  the  class  grade,  assuring 
that  students  had  adequate  incentive  to  participate  in  the  research  study  and  to 
llL  assignments.  Students  were  told  before  each  work  session 
that  all  members  of  their  group  would  receive  the  same  grade-the  grade  they 
arned  on  their  analysis  paper.  An  objective  of  the  researc .  was 
participation  patterns  that  occur  when  students  have  been  told  that  all  members  of 
ihe  group  will  receive  the  same  grade.  Therefore,  students  were  not  graded 
individually  for  these  assignments  even  though  the  ""d J* 
were  identified  and  recorded  during  the  study.   Each  group .met  three  times  for  75 
minutes  each,  to  work  on  three  different  case  studies.  All  of  the  group  meeting 
sessions,  including  those  in  the  Decision  Support.  Center  were  videotaped.  The 
video  camera  was  set  up  in  advance  and  was  left  unattended  in  full  view  of  the 
students.  In  the  face-to-face  group  meetings,  only  the  four  members  of  he  group 
were  present  in  the  classroom  during  the  group  meeting.  In  the  computer-mediated 
group  meetings,  one  or  two  system  administrators  were  present  in  the  Decision 
Support  Center  room  with  the  four  group  members.  The  system  administrators 
were  present  solely  to  facilitate  use  of  the  electronic  meeting  system  software, 
staying  in  the  background  as  much  as  possible. 
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differences  among  ,he  case  "tS.  in  |Z,  S        ?  "ny  offMt "»"*b«*d  by 
knowMp  and  bS  SlSfS^rf«J55l&^  ■SJ"*'^  ■  . 

repor,  to"^  group  prepared  a  hand-writ.en 

into  scripts  to  facilitate  scor iZ  of face-Io-faco  meehngs)  wore  transcribed 

SSKKStSLE7  ZEE? T  T- ,h" — t*B 

complete am.cneeo  Thll  ?„  "T  n.1"' nr"'  i"'livi,lu"'  h"''  n«  "P"-^  « 
and  0,0  'Sr'  ',l""nR  <,,0rh""»  '«h  f«  >  ~* 


Result 
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A  McHt  of  dependent  observations  was  used  on  both  composite  groups  to 
compare  the  difference  in  Ihe  number  of  T- 1  comments  (direct  contributions  to  the 
assigned  task).  The  experimental  group  showed  a  significant  increase  (p<-05)  in  the 
mean  number  of  T- 1  contribut  ions  in  the  computer-mediated  sessions  over  the  face- 
to-face  sessions.  No  significant  difference  was  found  in  the  mean  number  of  Tl 
contributions  between  the  t  wo  successive  face-to-face  sessions  in  the  control  group. 

A  1-test  of  paired  variances  was  used  to  compare  the  variance  of  All  T 
comments  (all  task-oriented  comments)  contributed  within  each  group.  There  was  a 
significant  ly  smaller  variance  (p<.()5)  in  the  number  of  All  T  comments  made  in  the 
experimental  group's  computer-mediated  session  than  in  its  face-to-face  session.  No 
significant  difference  was  found  between  the  variance  of  All  T  comments  made  in 
the  control  group's  two  face-to-face  sessions. 

Summary  of  Comment  Category  Scores  for  Experimental  and  Control  liim® 


GROUP  1:  Treatment  Group 

Duration:  47  minutes 
Face-to-Face  Communication 
Student   T-1A11T  All  G    A11S  Total 
1 
2 
3 
4 


5 

37 

7 

3 

47 

19 

19 

17 

61 

46 

2 

109 

13 

13 

11 

63 

24 

5 

92 

17 

18 

I 

11 

37 

11 

1 

49 

10 

12 

1 

44 

198 

88 

11 

297 

59 

62 

2 

15% 

67% 

30% 

4% 

100% 

92% 

97% 

3% 

Durat  ion;  50  minutes 
Computer-Mediated  Communication 
Tl     AUT    AUG    A1IS  Total 

19 
13 
19 
13 


0% 


64 

100% 


29  T  comments  interrupted 


GROUP  2:  Treatment  Group 

Duration:  64  minutes 
Face-to- Face  Communication 


Student 

Tl  AUT 

All  (3 

A11S 

Total 

T-l 

A11T 

1 

0  2 

1 

3 

29 

29 

2 

18  102 

29 

9 

140 

35 

38 

3 

27  107 

48 

5 

160 

37 

37 

4 

15  95 

76 

7 

178 

22 

22 

60  306 

154 

21 

481 

123 

126 

12%  64% 

32% 

4% 

100% 

94% 

96% 

31  T  comments  interrupted 


Duration:  62  minutes 
Computer -Mediated  Com m un icat  ion 
All  G  All  S  Total 
29 
42 
37 
23 


3 


3 

2% 


1 


1 


2  131 
2%  100% 


GROUP  3:  Treatment  Group 

Duration:  31  minutes 
Pace-to- Face  Communication 


Student 
1 

2 
3 
4 


T-l 
17 
14 

5 
9 

45 
21% 


All  T 
51 

36 
41 
33 


All  G 
10 
15 
8 
7 


AJ1S 
3 
1 
4 
2 


Total 
64 

52 
53 
42 


161 

76% 


40 
19% 


10 

5% 


211 

100% 


Duration:  50  minutes 
Computer-Mediated  Communication 


T-l 
51 

36 
15 
27 

129 

93% 


A11T 

52 
37 
16 
27 


All  G    All  S 


132 

96% 


6 

4% 


0 

0% 


Total 
52 
37 
22 
27 

138 
100% 


19  T comments  interrupted 
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GROUP  4:  Control  Group 

Duration:  76  minutes 
Face-to-Face  Communication  #1 


Duration:  55  minutes 


Student 
1 

2 
3 
4 


T-I 
15 
1 

23 
10 


A11T 

75 
7 
92 
41 


All  G 
36 
1 

21 
10 


All  S 


Total 
111 

8 
120 
52 


49    215      68         H  291 
17%   74%    23%       3%  100% 
28  T  comments  interrupted 

GROUP  5:  Control  Group 

Duration-  62  minutes 
Face-to-Face  Communication  #1 
Student    T-1  All  T  All  G    All  S  Total 
1 
2 
3 
4 


6 

87 

17 

9 

1  13 

16 

67 

24 

1 1 

102 

4 

35 

10 

5 

50 

28 

128 

70 

1 1 

209 

54 

317 

121 

36 

474 

Kace-to-Face  Communication  #2 

T-I  AUT 

AUG 

All  S  Total 

16  48 

.3 

16  67 

10  31 

2 

17  50 

12  41 

16 

18  75 

1 1  36 

10 

9  55 

49  156 

31 

60  247 

20%  6.3% 

13% 

24%  100% 

16  T 

comments  interrupted 

Duration:  48  minutes 

Face-to- Fare  Communication  #2 

T-I     All  T 

All  G 

All  S  Total 

4  m 

II 

32  109 

21  87 

22 

20  129 

6  28 

12 

10  50 

13  78 

.39 

30  147 

11%  67%    26%       8%  100% 
37  T  comments  interrupted 


44 
10% 


259       84        92  4.35 
60%      19%     21%  100% 
55  T  comments  interrupted 

Comment  Category  Codes: 

T-l  -  Comments  that  made  a  direct  contribution  to  the  task  requirements 

(T-l  is  a  subcategory  of  All  T) 
A11T-  Task-oriented  comments  that  related  directly  to  the  analysis  task 
All  G-  Group  process  comments  that  directed  group  effort  or  provided  feedback 
All  S-  Social  and  other  non  task-related  comments 

Discussion 

,         Th°  rf  ults  of  this  8tuuv  swrted  the  authors'  expectations  regarding  both 
hypotheses^  Students  were  more  productive  in  accomplishing  task  objectivran 
parucpated  more  ur;  formly  when  working  together  using  computer  meSed 
communications.  A  larger  study,  using  approximately  ninety  students  is  planned 
for  spnng  semester  1993  to  follow  up  on  the  promising  results  of  this  study 
Several  poss.ble  reasons  for  the  increase  in  student  productivity  are 
suggested  by  the  data  summarized  in  Table  1.  There  are  almost  no  group  process  or 
social  interactions  between  students  who  are  communicating  th^ghtT  EZZSZ 
network,  resulting  in  a  more  task-focused  work  session.  The  deperSnabzeZaSre 

nteraction^  Another  possible  reason  for  the  increase  in  productivity  may  be  related 

£Sta3?  ^ ™;S;hat,!aSk-<0riCn,(,d  COmmentS  W6re  -terrupt^d  b/le^nf1 
else  in  the  group.  The  number  of  interrupts  per  session  ranged  from  16  to  55  with  a 
mean  of  approximately  31.  Group  process  and  social  comments  were  interred 

haTan" 2K  H     '       ^  '^"T*  t0  T  C°mmentS  reP«~      numberTf  S.es 
that  an  mdiv.dual  was  unsuccessful  in  ge.t  ing  out  an  idea  that  was  task-oriented 

The  compet.Uon  for  the  opportunity  to  speak  in  face-to-face  session"  that is 
evidenced  by  the  mterrupts  may  also  provide  a  clue  as  to  why  not  all  s.ude »T 
parfcpate  equally  m  collaborative  problem -solving  sessions  Students  who  are  less 
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insistent  on  being  heard  will  participate  less.  In  a  situation  where  there  is  not 
competition  for  air  time,  such  as  in  sessions  conducted  using  computer-mediated 
communications,  those  same  students  may  dramatically  increase  their  level  of 
participation.  Similarly,  students  who  like  to  assume  a  leadership  role  in  face-to- 
face  meetings  find  it  more  difficult  to  control  and  lead  a  group  when  they  are 
communicating  electronically. 

Computer-mediated  communications  can  function  effectively  as  an 
alternative  means  of  teaching  collaborative  problem  solving,  particularly  when  the 
instructor  would  like  to  be  able  to  grade  individual  student's  contributions.  The 
benefit  of  increasing  task-focused  behavior  by  students  is  elso  evident  The  authors 
suggest  the  use  of  computer-mediated  communications  as  a  supplement  to 
traditional  face-to-face  problem-solving  sessions,  not  as  a  replacement.  Valuable 
learning  and  teamwork  experiences  are  embedded  in  the  social  and  group  process 
interactions  that  make  up  a  largo  share  of  the  communication  in  face-to-face 
meetings. 
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Qualitative  Research?  Quantitative  Research?  Whaf  s  the  Problem? 
Resolving  the  Dilemma  via  a  Postconstructivist  Approach 


Introduction 


The  -qualitative  vs  quantitative-  debate  has  been  part  of  the  fabric  of 
educational  research  for  some  time.  (  cf.  Smith,  1983;  Gage.  1989;  Rizo, 
1991;  Salomon.  1991  for  various  discussions  of  the  debate)  At  stake  in  this 
debate  is  the  issue  of  whether  two  worldviews  of  research  can  coexist  or  if 
one  has  to  eventually  either  incorporate  or  discredit  the  other.  Most  of  the 
various  papers  on  this  debate  have  been  fmmed  at  the  methodological  level  of 
discussion.  In  this  paper,  I  would  like  to  take  a  slightly  different  turn.  I 
would  like  to  argue  that  it  is  more  useful  to  look  at  this  debate  in  terms  of  the 
two  overarching  philosophical  positions  that  ground  the  debate  over  methods. 
In  short,  I  would  like  to  show  that  the  -qualitative  vs.  quantitative"  debate, 
Played  out  in  foundational  terms  for  educational  researchers,  is  actually  a 
constructivism  vs.  positivism  argument 

When  we  look  at  current  formulations  of  qualitative  methods  in 
educational  research,  more  traditional  notions  of  straightforwardly  applying 
ethnographic  and  other  anthropological  and  sociological  techniques  have  been 
more  or  less  replaced  with  a  call  to  approaching  qualitative  research  from  a 
constructivist  perspective,  (cf.  Cuba,  1992;  Eisner,  1992;  Duffy  & 
Jonassen.1991)  Furthermore,  most  issues  that  surface  in  an  examination  of 
constructivism  are  actually  n&sw^  to  issues  that  are  found  in  the  larger 
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qualitative  context,  and,  given  the  emerging  prominence  of  constructivism  in 
educational  terhnology  research,  an  analysis  of  constructivism  per  se  ought  to 
be  concretely  fruitful  to  the  field.   Positivism  has  arguably  been  the 
philosophical  basis  for  most  quantitative  research  in  education,  (cf.  Schrag. 
1992;  for  a  recent  defense  of  positivism  in  educational  research)  and  so  is 
the  perfect  philosophical  counterpoint  for  constructivism 

When  we  compare  constructivism  and  positivism  I  hope  to  show  that 
there  are  actually  two  different  things  that  are  meant  when  the  term 
-constructivism  is  used,  and  this  makes  any  simple  attempt  at  reconciling  the 
larger  schism  between  constructivism  and  positivism  nearly  impossible. 
Furthermore,  1  hope  to  demonstrate  that  the  schism  in  constructivism  is  a 
function  of  an  older  schism,  and  that  in  order  to  move  beyond  the  framework 
provided  by  positivistic  thinking,  we  must  adopt  one  of  the  senses  of 
constructivism  over  the  other.  And  finally.  I  am  going  to  propose  that  since 
the  -wrong"  version  of  constructivism  is  the  sense  most  currently  in  usage  by 
educational  researchers,  that,  starting  now,  we  adopt  the  label  of 
-postconstructivism"  to  distinguish  the  -good"  version  of  constructivism 
from  its  "evil  twin." 

Constructivism  and  Educational  Technology  Research 

Let  me  start  off  by  saying  that  some  of  my  best  friends  are  construct ivists. 
I  also  would  like  to  acknowledge  that  constructivism  draws  much  of  its  energy 
from  the  same  recent  historical  trends  that  have  come  to  produce  what  we 
consider  to  be  mainstream  qualitative  research  in  educatioa  including 
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holistic  ethnography,  symbolic  interactionism,  cultural  anthropology,  and  the 
like.  (cf.  Jacob.  1987.  for  a  review  of  these  trends)  However.  I  think  it  is 
important  for  us  to  realize  that  most  constructivist  practitioners  assume 
certain  stated  and  unstated  assumptions  that  ground  it  very  clearly  to  certain 
trends  of  thought  and  that  unreflectively  accepting  the  basic  tenets  of  these 
trends  can  lead  us  into  some  troubling  consequences. 

Merrill  ( 1991)  summarizes  constructivism  as  being  based  on  the 
following  assumptions;  1)  knowledge  is  constructed  by  the  learner.  2) 
learning  is  a  personal  interpretation  of  experience.  3)  learning  is  active.  4) 
learning  is  collaborative.  5)  learning  is  situated  in  real  world  contexts,  and 
6)  assessment  of  learning  is  integrated  within  the  learning  context  itself,  (p. 
46)  Duffy  &  Jonassen  (1991)  acknowledge  that  "constructivism  provides  an 
alternative  epistemological  base"  (p.  8)  for  research  in  educational 
technology.   They  cite  recent  work  in  situated  cognition  as  the  basis  for 
much  of  constructivist  theory,  (cf.  Brown.  Collins  &  Duguid.  1989  for  a 
statement  of  the  basic  principles  of  situated  cognition  in  educational 
research)  contrasting  such  theoretical  work  against  what  Lakoff  &  Johnson 
( 1980)  called  the  "Objectivist"  tradition  When  we  look  at  the  Objective 
tradition,  we  find  that  it  is  really  what  we  have  come  to  call  the  positivist 
position  in  research  Therefore,  it  is  no  surprise  that  constructivism  is 
deliberately  anti-positivistic  in  its  outlook 

While  constructivism  appears  to  differ  from  older,  behavioristic  models  ol 
research,  it  seems  to  be  perfectly  attuned  to  more  current  directions  in 
psychology.  For  instance,  Perkins  (1991)  explicitly  links  constructivism  with 
information  processing  theory  by  noting: 
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information  processing  models  have  spawned  the  computer  model  of 
the  mind  as  an  information  processor.  Constructivism  has  added  that 
this  information  processor  must  be  seen  as  not  just  shuffling  data,  but 
wielding  it  flexibly  during  learning  -  making  hypotheses,  testing  tentative 
interpretations,  and  so  on.  (p.  21) 

By  linking  information  processing  and  constructivism,  Perkins  brings 
together  contemporary  cognitive  theory  and  constructivist  theory.  This  is  an 
important  step,  because  it  anchors  constructivism  within  the  mainstream  of 
scientific  thought  in  the  human  sciences.  Spiro,  et  al  (1991)  continue  this 
thread,  by  further  linking  constructivism  with  schema  theory.  With  its 
insistence  on  being  more  free-form  and  less  centralized,  constructivist  theory 
also  has  a  natural  affinity  to  theoretical  work  in  coi^ectionism.  (Bereiter. 
1991)  hypermedia  (Tolhurst,  1992)  and  multimedia  (Dede.  1992)  On  a 
practical  educational  level  Bagley  &  Hunter  (1992)  argue  that  constructivism 
is  the  "glue"  that  can  hold  together  the  combination  of  technological 
innovation  and  restructuring  needed  in  our  school  systems. 

All  else  aside,  the  debate  about  constructivism  centers  on  whether  we  are 
willing  to  take  a  positivist  stance  toward  research  in  education  as  was  done  in 
the  past,  or  whether  we  need  a  new  position.  In  order  to  make  such  a 
decision,  the  researcher  should  be  aware  of  what  a  position  entails,  and  where 
it  came  from.  Therefore,  we  need  to  focus  on  where  constructivist  ideas 
come  from,  and  why  they  have  such  purchase  on  the  imagination  of 
researchers  in  education  and  other  human  sciences. 
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The  Roots  of  Constructivism 


What  are  the  basic  tenets  of  constructivism?  Cole  ( 1992)  claims  that 
-social  negotiation  of  meaning"  and  -critical  argumentation"  (p.  27)  are  the 
keystones  to  the  constructs  approach.  Cunningham  ( 199 1)  in  his  neo- 
Galilean  dialogue  between  a  constructivi st  and  a  positivist,  has  "Salvia*' 
define  constructivism  thusly: 

Constructivism  holds  that  learning  is  a  process  of  building  up  structures 
of  experience.  Learners  do  not  transfer  knowledge  from  the  external 
world  to  their  memories;  rather,  they  create  interpretations  of  the  world 
based  on  their  past  experiences  and  their  interactions  in  the  world 
How  someone  construes  the  world,  their  existing  metaphors,  is  at  least 
as  powerful  a  factor  influencing  what  what  is  learned  as  any  characteristic 
of  that  world  Some  would  even  argue  that  knowledge  that  is 
incompatible  with  or  unaccounted  for  in  an  individual's  interpretation 
cannot  be  learned  (p.  13) 

These  ideas  are  quite  different  from  the  notions  that  have  guided 
educational  research  through  most  of  the  twentieth  century.    What  are  their 
historical  roots?  All  talk  of  cognitive  theory  and  methodology  aside,  it  is 
reasonable  to  claim  that  constructivism  is  basically  grounded  in  the  work  of 
three  contemporary  philosophers  of  inquiry  methodology  -  Richard  Rorty. 
Nelson  Goodman,  and  Paul  Feyerabend 
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Richard  Rorty,  with  his  neo-pragmatism,  (Rorty,  1982;  Trimbur  &  Holt 
1992)  is  probably  the  most  influential  person  in  the  philosophy  of  science 
today,  as  well  as  being  the  heir  apparent  to  Kuhn's  tremendously  important 
theorizing  on  the  role  of  paradigms  and  paradigm  shifting  in  scientific 
research.  Rorty  pushes  the  idea  of  the  paradigm  shift  to  the  extreme  with  his 
model  of  narrative  philosophy.  In  narrative  philosophy. 

Rorty  shifts  the  terms  from  a  search  for  method  to  a  strategy  of 
historicization  He  urges  us  to  look  at  philosophical  discourse  as 
language-in-action  -  not  a  privileged  perspective  that  holds  out  the  hope 
of  finally  moving  from  speculation  to  science  but  simply  the  stories 
philosophers  tell  each  other...  To  historic ize  philosophy  in  this 
way...means  we  will  no  longer  possess  a  universal  measure  by  which  to 
judge  knowledge  claims  and  the  accuracy  with  which  they  represent 
reality...  To  recognize  that  philosophy  is  no  more  and  no  less  than  its 
own  narratives  is  to  recognize  that  we  will  always  get  the  past  that  we 
deserve  -  not  a  final  definitive  version  of  philosophical  problems  and 
issues  but  the  version  our  present  situations  calls  up.  We  will  have  what 
was  there  all  along  -  ourselves,  our  common  activities,  our  attempts  to 
cope  with  reality.  And  we  will  be  no  worse  for  our  judgements.  It  is  just 
that  our  judgements  will  no  longer  be  concerned  with  what  is  true  and 
accurate  according  to  some  extrahistorical  criteria.  (Trimbur  &  Holt 
1992:  75) 
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Nelson  Goodman's  Ways  of  Worl ^mflking  (1978)  reads  like  a  manifesto  for 
the  constructivist  movement 

Truth,  far  from  being  a  solemn  and  severe  master,  is  a  docile  and 
obedient  servant  The  scientist  who  supposes  that  he  is  single-mindedly 
dedicated  to  the  search  for  truth  deceives  himself.  He  is  unconcerned 
with  the  trivial  truths  he  could  grind  out  endlessly....  He  seeks  system, 
simplicity  and  scope;  and  when  satisfied  on  these  scores  he  as  much 
decrees  as  discovers  the  laws  he  sets  forth,  as  much  designs  as  discerns 
the  patterns  he  delineates...  Truth,  moreover,  pertains  solely  to  what  is 
said,  and  literal  truth  to  what  is  said  literally.  We  have  seen,  though,  that 
worlds  are  made  not  only  by  what  is  said  literally  but  also  by  what  is  said 
metaphorically....  (p.  18) 

Paul  Feyerabend's  (1975)  self-styled  anarchistic  emphasis  on  the  practice 
of  science  as  opposed  to  theorizing  also  strikes  a  fundamental  chord  in  the 
constructivist  model: 

It  is  clear,  then,  that  the  idea  of  a  fixed  method,  or  of  a  fixed  theory  of 
rationality,  rests  on  too  naive  a  view  of  man  and  his  social  surroundings. 
To  those  who  look  at  the  rich  material  provided  by  history,  and  who  are 
not  intent  on  impoverishing  it  in  order  to  please  their  lower  instincts, 
their  craving  for  intellectual  security  in  the  form  of  clarity,  precision, 
'objectivity,  'truth',  it  will  become  clear  that  there  is  only  one.  principle 
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that  can  be  defended  in  ail  circumstances  and  in  ajl  stages  of  human 
development  It  is  the  principle:  anything  goes.  (  P.  28,  italics  his). 

Cuba  (1992)  explicitly  links  Feyerabend  to  his  project  of  relativism  which 
Guba  says  "  ..is  an  essential  element  in  defining  the  ontological  and 
epistemological  presuppositions  of  the  constructivist  inquiry  paradigm  of 
which  I  am  a  proponent"  (p.  17)  Eisner,  (1992)  whose  critical  version  of 
qualitative  methodology  is  akin  to  Cuba's  model  of  constructivism,  draws 
heavily  on  Rorty  and  Goodman  in  his  attempt  to  critique  objectivity  in  a 
postpositivist  frame.  Therefore,  we  can  assume  that  Rorty,  Goodman,  and 
Feyerabend  are  central  figures  in  the  emerging  constructivist-qualitative 
paradigm  developing  and  growing  within  educational  research. 

What  are  the  common  issues  that  unite  these  three  foundational 
constructivist  thinkers  ,  and  how  can  we  examine  those  issues  to  help 
understand  the  basic  tenets  of  constructivist  thinking?  Essentially.  Rorty, 
Goodman,  and  Feyerabend  all  share  a  concern  for  the  historical  side  to  human 
inquiry,  and  this  is  no  accident  This  is  because  all  three  of  them  see 
themselves  as  members  of  the  pragmatist  tradition,  a  tradition  that  Pepper 
(1942),  in  his  review  of  metaphysical  positions,  labeled  "contextualism". 
Contextualism.  furthermore,  took  its  root  metaphor  from  the  idea  of  the 
historical  event  Pragmatism  has  always  looked  to  the  context  of  practice,  in 
historical  and  cultural  terms,  as  the  basis  for  resolving  issues  of  meaning,  (cf. 
House.  1991;  Cherryholmes.  1992;  House.  1992  for  a  discussion  of  the  role 
of  pragmatism  vs  realism  in  scientific  inquiry  in  educational  research) 
Therefore,  we  can  conclude  that  constructivism,  as  articulated  in  educational 
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research  settings,  is  a  species  of  pragmatism.  This  link  to  pragmatism  will 
ironically  provide  us  with  the  means  to  move  from  constructivism  to 
postconstructivism.  since  we  will  make  that  move  by  Unking  the  concerns 
expressed  in  constructivism  to  the  ideas  of  the  philosophers  who  formulated 
pragmatism  in  the  first  place. 


Steps  toward  a  Postconstructivist  Model 


The  path  to  a  postconstructivist  model  of  educational  research  is  based 
on  the  split  within  pragmatism  that  has  shaped  that  philosophical  discipline 
from  its  outset  Pragmatism  is  a  movement  with  two  founders  -  an  actual 
founder  (C.S.  Peirce)  and  an  expedient  founder  (William  James).  Many  of  the 
problems  that  arise  in  pragmatism  are  based  on  the  fact  that  the  ideas  of  the 
actual  founder  and  the  expedient  founder  are  different  in  important  ways. 

Let  us  start  with  the  expedient  founder  of  pragmatism  -  William  James. 
In  1898.  in  a  lecture  at  the  University  of  California.  James  stated  his  version 
of  pragmatism  in  such  a  way  that  the  ideas  caught  the  imagination  of  the 
American  intellectual  community  for  decades  to  come.  Even  though  James 
was  careful  to  acknowledge  that  his  good  friend.  Charles  Peirce.  was  the 
founder  of  pragmatism,  few  people  actually  looked  at  Peirce's  ideas  Instead, 
they  followed  James's  dicta  James's  version  of  pragmatism  is  fairly  simple. 
He  held  that  pragmatism  was  a  means  for  determining  the  truth  of  certain 
claims.  That  method  works  as  follows  (James,  1948): 
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-Grant  an  idea  or  belief  to  be  true."  it  says,  "what  concrete  difference 
will  its  being  true  make  in  any  one's  actual  life?  How  will  the  truth  be 
realized?  What  experiences  will  be  different  from  those  which  would 
obtain  if  the  belief  were  false?  What,  in  short,  is  the  truth's  cash-value  in 
experiential  terms?" 

The  moment  pragmatism  asks  this  question,  it  sees  the  answer  True. 
jdm  gre  those  that  we  can  assimilate,  validate,  corroborate  and  YCrifr- 
False  ideas  are  "™*g  we  cannot  That  is       Practical  dtffcrence  it  makes 
for  us  to  have  true  ideas;  that,  therefore,  is  the  meaning  of  truth,  for  that 
is  all  that  truth  is  known-  as.  (pp.  160- 161;  italics  his) 

Now,  let  us  compare  James's  maxim  on  the  pragmatic  notion  of  truth 
with  the  Pragmatic  Maxim.  This  Maxim,  first  published  in  1878  and  pretty 
much  ignored  thereafter,  is  Peirce's  definition  of  pragmatism  (Peirce,  1955): 

Consider  what  effects,  that  might  conceivably  have  practical  bearings,  we 
conceive  the  object  of  our  conception  to  have.  Then,  our  conception  of 
these  effects  is  the  whole  of  our  conception  of  the  object  (p.  31) 

Granting  that  the  above  quote  is  hard  to  understand,  let  us  look  at  a 
passage  where  Peirce  explicates  his  maxim: 

...the  whole  meaning  of  an  intellectual  predicate  is  that  certain  kinds  of 
events  would  happen,  once  in  so  often,  in  the  course  of  experience, 
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under  certain  kinds  of  existential  conditions  -  provided  it  can  be  proved 
to  be  true.  (  p.  273;  italics  his) 

And  elsewhere: 

But  of  the  myriads  of  forms  into  which  a  proposition  may  be  translated, 
what  is  that  one  which  maybe  called  its  very  meaning?  It  is.  that  form 
in  which  the  proposition  becomes  applicable  to  human  conduct  not  in 
these  or  those  special  circumstances,  nor  when  one  entertains  this  or 
that  special  design,  but  that  form  which  is  most  directly  applicable  to 
self-control  under  every  situation,  and  to  every  purpose.  That  is  why  he 
locates  the  meaning  in  future-time;  for  future  conduct  is  the  only  conduct 
that  is  subject  to  self-controL  (p.  261) 

Finally,  let  us  turn  to  Murphy's  (1990)  paraphrase  of  Peircc's  pragmatic 
maxim,  which  Murphy  calls  Peirce's  Principle  of  Meaning: 

If  one  can  define  accurately  all  the  criteria  governing  uses  to  which  a 
predicate  can  be  put.  one  will  have  therein  a  complete  definition  of  the 
meaning  of  what  it  predicates,  (p.  46) 


Compare  this  now  with  Murphy's  (1990)  paraphrase  of  James's  theory  of 
truth: 
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What  is  true  in  our  way  of  thinking  is  the  production  of  beliefs  that  prove 
themselves  to  be  good,  and  good  for  definite,  assignable  reasons,  (p.  57) 

Peirce  and  James  mean  two  very  different  things  by  pragmatism  Peirce 
saw  the  Pragmatic  Maxim  as  a  way  of  determining  the  logical  nature  and 
consequences  of  certain  beliefs  in  terms  of  the  practical  consequences  of 
holding  those  beliefs.  James,  on  the  other  hand,  saw  the  Pragmatic  Maxim  as 
a  method  of  determining  the  truth  of  a  given  proposition.  For  Peirce,  belief 
claims  were  contextual  and  provisional,  yet  truth  claims  were  more  general 
and  logicaUy  accessible.  For  James,  truth  claims,  not  just  meaning  claims, 
were  provisional  and  contextual  Because  Peirce  held  that  there  can  be  truth 
over  and  above  what  we  might  think  to  be  true,  he  is  ultimately  a  realist 
Since  James  always  grounds  truth  claims  in  contextualist  activities,  he  is 
ultimately  a  relativist  And  the  gap  between  the  two  is  quite  vast 

The  pragmatism  that  informs  Rorty,  Goodman,  and  Feyerabend,  and  thus 
ultimately  the  constructivists,  is  derived  from  James.  For  instance,  Rorty 
(1982)  turns  explicitly  to  James  to  make  the  following  points  about  James's 
view  of  truth; 

James's  point  was  that  there  is  nothing  deeper  to  be  said:  truth  is  not 
the  sort  of  thing  that  h£§  an  essence....  Those  that  want  truth  to  have  an 
essence  want  knowledge,  or  rationality,  or  inquiry,  or  the  relation 
between  thought  and  its  object  to  have  an  essence.  Further,  they  want  to 
be  able  to  use  their  knowledge  of  such  essences  to  criticize  views  they 
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take  to  be  false,  and  to  point  the  direction  of  progress  toward  the 
discovery  of  more  truths.  James  thinks  these  hopes  are  in  vain.  (  p.  162) 

And  this  version  of  understanding  the  nature  of  truth  is  at  the  very  heart 
of  the  constructivist  program.  As  we  saw  earlier,  the  constructivist  holds  that 
we  construct  knowledge.  What  is  knowledge  but  those  things  that  we  hold  to 
be  true?  Furthermore,  the  constructivist  rejects  the  idea  that  there  is  a 
single  foundation  for  basing  the  pursuit  of  truth  on.  and  so  the  question  of 
truth  becomes  instead  an  exercise  in,  what  we  see  over  and  over  again,  the 
"social  construction  of  reality." 

Are  we  doomed  to  this  project  of  making  multiple  realities,  of  just  telling 
each  other  stories,  and  letting  the  question  of  truth  be  settled  by  whose  story 
is  more  interesting,  or  whose  will  is  stronger?  If  we  adhere  to  the  Jamesean 
perspective,  then  this  seems  to  be  the  case.  But  there  is  an  alternative  for 
inquiry  -  to  return  to  Peirce's  initial  statements  and  take  them  in  a  different 
direction 


Postconstructive  Research  -  A  Vision 


We  are  at  last  finally  free  to  talk  about  qualitative  research  in  general,  and 
thereby  finally  address  the  question  outlined  in  the  title.  Constructivist 
theory  has  made  significant  inroads  in  the  educational  research  community  by 
offering  a  vision  of  research  and  theory  that  seems  substantially  different  from 
the  prevailing  positivism  vision  based  on  more  traditional  models  of  scientific 
inquiry.  Constructivism  appeals  to  the  desire  of  many  in  the  field  to  be  freed 
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from  the  constraints  of  a  positivistic  mode!  of  inquiry,  (ct  Glesne  &  Peshkin. 
1992;  Laney,  1993)  It  cannot  be  over-emphasized,  though,  that  the 
fundamental  point  of  constructivism  is  to  deliberately  refute  positivistic 
positions.  By  taking  this  tack,  constructivist  ideas  are  necessarily  grounded  m 
the  same  problematic  that  the  positivist  tradition  was  grounded  in;  namely,  a 
concern  for  establishing  the  proper  conditions  for  gathering  and  using 
knowledge.  This  grounding  in  the  pursuit  of  knowledge  characterizes  the 
basic  task  of  modernist  thinkers.  (Lyotard  1979)  But  the  modernist 
problematic  is  no  longer  appropriate.  The  predicament  faced  by  most 
constructivists  is  that  they  sense  a  need  to  shift  beyond  a  modernist 
perspective,  but  they  attempt  to  do  so  by  addressing  the  matter  in  a 

modernist  fashion. 

There  is  no  escaping  the  fact  that  we  are  now  living  in  a  postmodern 
world  Lyotard  (Lyotard  1979;  Johnson,  1992)  has  declared  that  the 
postmodern  condition  is  based  on  the  notion  that  there  is  no  longer  a  -grand 
narrative'  that  defines  a  single  vision  for  culture.  We  have  to  abandon  the 
search  for  a  single  perspective  through  which  we  can  formulate  a  single  best 
way  of  doing  inquiry.  Gergen  (1992)  shows  us  that  postmodernism  involves  a 
turning  away  from  the  Enlightenment  with  its  insistence  on  individual 
cognition,  rational  order,  and  correspondence  models  of  inquiry,  toward  a 
new  sensibility  of  collective  rationality,  action,  and  examination  of  phenomena 
as  "texts-  instead  of  repositories  of  fact  or  instantiations  of  thecy.  Thinkers 
such  as  Heidegger  (Guignon,  1992)  and  Baudrillard  (1988)  have  emphasized 
that  language  and  its  rhetorical  nature  play  a  crucial  role  in  the  ways  that  we 
understand  our  world  thereby  clarifying  the  -textness"  of  the  world 
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Both  James  and  Peirce  would  appreciate  the  move  in  inquiry  towards  the 
postmodernism  described  above.   But  their  characterizations  of  the  situation 
would  be  quite  different  James  leads  us  away  from  the  pole  of  strict 
objectivity  by  positing  a  relativist  and  ultimately  subjective  model  of  truth  and 
therefore  inquiry,  while  Peirce  rejects  the  subjective  vs  objective  duality 
altogether.  It  is  this  rejection,  which  is  far  more  radical  than  any  project  that 
James  or  the  constructivists  would  suggestthat  finally  allows  us  to  build  the 
bold  and  radical  vision  of  qualitative  research  that  education  needs  in  order  to 
function  properly  in  a  postmodern  setting. 

As  my  fundamental  move  in  this  paper,  I  would  like  to  show  how  Peirce 
gets  out  of  the  subjective-objective  trap,  and  then  show  the  consequences  of 
this  move  as  the  basis  for  a  genuinely  qualitative  model  of  educational 
research,  one  that  does  not  slip  either  into  the  trap  of  an  -anything  goes" 
relativistic  subjectivism  or  the  -just  the  facts"  pseudo-scientific  objectisitic 
positivism  that  has  strangled  the  vitality  from  educational  research.  (  cf. 
Shank.  1987;  Shank.  1988;  Shank,  1990;  Shank.  1992;  Shank  & 
Cunningham,  1992) 

The  key  to  creating  a  pragmatically-informed  model  of  inquiry  is  to 
realize  that  pragmatism,  as  Peirce  conceived  of  it  was  a  totally  radical 
departure  from  me  Enlightenment  scenario.  That  scenario  was  established 
by  the  work  of  Descartes.  In  point  of  fact  the  subjective  -  objective  split  that 
haunts  modernism  is  a  direct  consequence  of  the  Cartesian  method  of 
inquiry,  known  as  the  method  of  doubt 

Briefly,  the  method  consists  of  deliberately  doubting  the  truth  of 
anything,  until  we  arrive  at  something  we  cannot  doubt;  namely,  the  famous 
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-cogito  ergo  sum"  or  "I  think,  therefore,  1  am."  This  method  requires  both  a 
subjective  observer  and  the  objective  realm  of  the  observed.  This  method  also 
creates  an  unbridgeable  split  between  the  observer  and  the  observed,  keeping 
the  observer  always  out  of  the  very  reality  of  that  he/she  is  trying  to  observe. 
Peirce  realized  that  this  split  was  a  function  not  so  much  of  Descartes's 
intended  worldview,  but  of  his  method  instead.  So,  in  order  to  create  a  post- 
Cartesian  worldview,  one  that  is  free  from  the  subjective  -  objective  split  we 
need  to  create  a  new  method. 

To  make  a  long  story  short,  the  Pragmatic  Maxim  was  a  direct 
consequence  of  Peirce' s  efforts  to  replace  the  Cartesian  method.  Peirce 
realized  that  Descartes's  method  of  doubt  was  flawed  because  the  doubt  that 
Descartes  engendered  was  not  genuine.  Peirce  chose  to  focus  on  genuine 
feelings  of  doubt  that  we  feel,  and  the  means  that  we  try  to  alleviate  them  and 
move  back  into  states  of  belief.  This  fixation  of  belief  in  the  face  of  genuine 
doubt  was  the  basis  for  all  inquiry,  in  Peirce's  view.  So  long  as  we  have  no 
reason  to  doubt  either  our  senses  or  our  current  understanding  of  things,  says 
Peirce,  then  we  have  no  reason  not  to  suppose  that  they  are  true.  As  soon  as 
something  intrudes  into  our  current  state  of  understanding,  though,  we 
realize  that  we  Were  not  right  after  all,  and  that  truth  is  different  from  what 
we  had  held  it  to  be.  That  instance  from  experience,  furthermore,  is  a  clue 
that  the  way  that  we  looked  at  reality  was  not  quite  right,  and  by  that  fact 
alone,  we  now  have  a  new  clue  as  to  the  ultimate  nature  of  reality.  Through 
this  correctable  process,  held  Peirce,  we  co-evolve,  with  the  help  of 
experience,  a  significant  understanding  of  reality,  one  that  we  have  no  reason 
to  suppose  that  is  not  true,  but  one  that  we  cannot  feel  smugly  is  correct 


919 


ERIC 


What's  the  Problem? 

18 


Inquiry,  then,  becomes  an  act  of  "reading"  experience  to  gain  more  and 
more  clues  about  the  nature  of  reality.  We  treat  the  items  of  experiences  as 
neither  objective  facts  whose  truth  we  can  grasp  in  an  unmediated  way,  nor  as 
subjective  opinions  whose  truth  is  simply  a  matter  of  what  we  want  it  to  be. 
Rather,  we  treat  those  items  as  signs  of  reality.  In  order  to  read  them,  we 
have  to  live  in  the  gap  between  the  objective  and  the  subjective,  and  we  soon 
come  to  see  that  the  gap  as  illusory. 

Our  postconstructive  model  says  that  we  do  not  construct  reality,  because 
reality  itself  is  so  rich  and  significant  that  all  we  need  to  do  is  to  read  it  But 
we  need  to  enrich  our  current  conceptualization  of  -reading."  Reading  is  not 
a  process  of  extracting  information  from  a  printed  page,  nor  is  it  the  creation 
of  meaning.  Reading  and  writing,  telling  and  hearing,  and  narrating  and 
understanding,  are  all  the  same  process.  We  listen  to  and  read  stories  in  an 
attempt  to  try  to  understand  the  nature  of  reality  a  bit  better,  but  we  tell  and 
write  them  for  the  same  purpose.  To  say  that  we  read  and  write  to  get  access 
to  reality  sounds  silly,  but  to  say  that  we  write  and  tell  in  order  to  create 
reality  is  just  as  silly.  We  can  finally  argue  that  the  act  of  "reading"  will 
change  both  us  and  reality,  since  we  are  both  co-evolving.  We  have  the  tenets 
of  experience  to  keep  us  straight  and  the  Pragmatic  Maxim  to  keep  us  from 
slipping  into  models  of  inquiry  that  lead  us  away  from  reality. 

What  would  such  a  model  look  like  in  practical  terms?  It  is  legitimately 
postmodern,  in  that  it  realizes  that  there  is  no  one  way  to  look  at  reality.  But 
it  rejects  the  notion  of  dependence  on  the  observer  and  replaces  it  with  a 
model  of  the  inquirer  as  a  participant  When  we  construe  the  inquirer  as  a 
participant  we  see  that  he/she  is  not  looking  at  reality  from  some  outside  or 
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privileged  position,  but  instead  is  in  the  middle  of  reality.  In  fact,  Peirce 
would  claim  that  all  inquiry  begins  with  us  being  in  the  middle  of  some 
already  existing  world,  and  that  our  first  act  is  to  try  to  make  sense  of  that 
world  The  constructivist  would  say  that  we  find  ourselves  in  a  situation 
where  we  have  to  make  meaning  in  the  sense  of  imposing  meaning  on  a 
situation  which  has  no  meaning  until  we  put  it  there.  Remember  James's  old 
dictum  of  the  -buzzing,  booming  confusion"  we  experience  until  we  impose 
order  on  it?  Furthermore,  again  following  James,  the  order  that  we  impose 
then  becomes  true  so  long  as  that  truth  is  useful  to  us  to  maintain  the  order 
we  have  already  imposed.  Pretty  soon,  we  find  ourselves  in  the  dilemma  of 
being  in  a  situation  where  any  meaning  is  totally  a  product  of  what  we  have 
put  there.  What  does  the  world  become,  then,  besides  ourselves?  And  what 
is  instructional  design  under  the  aegis  of  this  thinking  but  the  use  of  one's 
intellectual  will  to  impose  a  worldview  on  others,  which  is  then  claimed  to  be 
true  because  it  conforms  to  the  wiU  of  the  design  creator?  And  what  does 
educational  technology  become,  then,  except  for  a  means  to  enhance  this 
imposition  process?  What  about  the  constructivisf  s  claim  that  the  creation  of 
knowledge  needs  to  shift  from  the  designer  and  instructor  to  the  student? 
What  is  this  other  than  the  shifting  of  the  will  to  either  the  individual  or  the 
collective  student?  At  its  very  best  constructivism  with  its  inherent 
relativism  and  dependence  of  the  subjective  stance  of  designers  and  students, 
becomes  a  way  to  shift  from  an  individual  to  a  collective  solipsism 

As  Peirce  pointed  out  in  his  famous  essay  on  the  fixation  of  belief,  the 
only  way  to  maintain  a  belief  which  we  do  not  want  to  abandon  is  to  hold  onto 
it  with  tenacity  and  to  make  sure  we  do  not  find  ourselves  in  any  kind  of 
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circumstantial  situation  where  that  belief  might  get  challenged  The  path  of 
constructivism  leads  inevitably  to  the  championing  of  the  will,  rather  than  the 
intellect  as  the  supreme  arbiter  on  what  counts  as  knowledge. 

The  postconstructivist  alternative  is  to  accept  that  we  cannot  impose  our 
will  freely  on  reality,  but  that  we  are  not  slaves  to  some  configuration  of 
reality.  It  accepts  the  fact  that  we  are  always  in  reality,  and  in  relation  to  all 
other  systems  in  reality.  And  that  every  experience  has  significance  as  a  part 
of  the  whole  picture  of  reality.  And  that  reality  is  not  constructed,  or  a  simple 
set  of  principles  underlying  a  complex  surface,  but  is  instead  intelligible  on  its 
own  terms.  But  we  accept  that  the  terms  of  intelligibility  of  reality  are  not 
directly  translatable  into  human  terms.  We  have  to  treat  experiences  as  signs 
of  the  nature  of  reality,  as  answers  to  questions  we  are  not  sure  how  to  ask. 
We  have  to  shift  our  model  of  the  inquirer  away  from  the  prospector  model  of 
the  positivist,  who  seeks  to  find  the  nuggets  of  truth  under  the  welter  of 
experience,  and  the  novelist  model  of  the  constructs  st,  who  seeks  to  create 
truth  out  of  what  seems  to  be  available.  The  postconstructivist  is  postmodern 
in  realizing  that  there  any  many  paths  of  inquiry,  but  unlike  the  constructs st 
the  postconstructivist  does  not  hold  that  we  impose  the  order  we  seek  by 
following  many  paths.  The  model  of  the  postconstructivist  follows  along 
semiotic  lines,  where  the  inquirer  realizes  that  experiences  are  signs  of  order 
and  understanding  and  meaning  and  that  they  have  consequences  not  just  in 
and  of  themselves  but  of  how  they  are  taken  as  signs.  The  model  for  the 
postconstructivist  is  that  of  a  detective,  treating  experiences  as  clues  and 
omens  and  symptoms  of  an  order  that  represents  a  rich  and  complicated 
reality  that  we  can  only  understand  within  a  process  of  participation  as  part  of 
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reality  itself.  The  postconstructivist  turns  away  from  knowledge  per  se  as  the 
starting  point  of  inquiry,  and  acknowledges  that  there  is  just  as  complex  and 
as  important  an  inquiry  based  on  increasing  understanding,  that  an  inquiry  of 
understanding  is  just  as  potentially  systematic  as  an  inquiry  of  knowing,  and 
that  any  real  inquiry  of  knowing  depends  on  an  inquiry  of  understanding. 

So  where  have  we  ended  up  on  this  process  of  going  beyond 
constructivism?  1  want  to  charge  that  educational  technology  is  in  a  unique 
position  of  being  able  to  help  further  an  inquiry  into  understanding.  By 
developing  new  technologies  and  new  tools,  we  can  help  students  not  just 
make  meaning  out  of  a  jumble  of  data,  but  to  look  at  their  experiences  in  a 
participatory  way  and  to  combine  and  test  patterns  and  clues,  and  to  derive 
consequences  from  these  clue  patterns,  and  to  push  forward  with  new  and 
unique  insights.  I  feel  that  this  was  the  intended  mission  of  constructivism  in 
the  first  place,  but  it  cannot  achieve  that  mission  unless  it  gives  up  the  notion 
that  the  order  it  seeks  is  simply  imposed  by  the  subjectivity  of  the  inquirer. 

Let  me  conclude  this  entire  piece  and  hopefully  bring  my  points  home  in 
a  concrete  way  by  telling  you  about  a  computer  program  that  John  Ross  and  I 
designed  to  help  illustrate  a  semiotically  informed  model  of  inquiry.  We 
called  the  program  ART.  which  is  short  for  the  Abductive  Reasoning  Tool. 
The  idea  behind  ART  is  that  there  are  potentially  a  myriad  number  of  insights 
that  can  be  drawn  when  we  juxtapose  two  statements  in  an  arbitrary  fashion. 
We  start  with  a  topic  statement,  which  the  person  types  in.  Then,  the 
computer  randomly  accesses  one  of  over  a  thousand  generic  wise  sayings, 
which  is  physically  juxtaposed  with  the  initial  topic  and  called  the 
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"reflection-  statement  The  whole  point  of  the  process  is  that  the  inquirer 
feels  a  compulsion  to  link  those  two  statements  as  a  single  meaningful  whole, 
and  that  such  a  link  will  give  the  inquirer  a  new  insight  into  the  topic.  We  felt 
that  the  random  linkage  allowed  the  inquirer  to  explore  different  avenues  for 
insights  than  he/she  might  ordinarily  pursue.  By  realizing  that  the  linkage  is 
hypothetical  and  under  no  burden  to  be  -true"  we  are  allowed  to  pursue  an 
inquiry  of  understanding  that  does  not  generate  any  new  knowledge.  We 

would  not  be  able  to  take  this  stance  from  a  constructivist  perspective,  since 

we  would  be  compelled  to  be  making  some  reality  claim,  or  showing  how  the 

linkage  we  create  is  in  some  sense  "true*. 

So,  I  turned  to  the  program  to  ask:  "How  can  postconstructivist  ideas 

help  clarify  the  qualitative  vs.  quantitative  debate?" 

Here  is  the  actual  reflection  statement  "Our  intelligence  wraps  in 

obscurity  the  simple  natural  truths  in  order  to  get  credit  for  them.* 
I  leave  the  task  of  drawing  insights  to  you. 
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Abstract 

narrator  and  on-?crcln .  ^Sr  ?n^rSr  »5S.  f  ?V  V™'  and  rc,ationshiP  between  the 
grade  science  classes  K ^ fll,^ ;,CVel  fi,m-  Twcnty  scvt>nth- 

class  viewed  a  16-minute  film  atou™ ^Aw  nl^T  °ffiVC  trcatmcnt  8r°uPs"  Each 
experiment  with  f^S^ld^^^^S111!6  and  carryinS  °u<  ™ 
narration  was  changed  to  create Versions  w  th ?    °  f°r  °ach  grouP'  but  ,hc 
information  in  story  or  direclfon  forma?  A  tori     fh I      'T  Pr™"li»S  'ho 

narrator  also  changed  the  XtonStotoiS,  &      g7"d  Prcsenta«™  '<>"™t  <>' »he 
comprehension  of  info  ?on ^S^d^X^  CharaClCr  Rcca» 

mental  effort,  interest,  and  confTdenTc dSnJ K       f-rcP"«ed  amount  of  invested 
learned  more  when  the  InSI^SlTr'"  R0SU,tS  indicaled  studt>I1ts 
was  their  own  age.  Females' nd  catod  a  hK  SL  ? ■ 7  format',csPccia"y  when  the  narrator 
doing  plant  expfrimcnTl  Z£% StwIS  "J1*  fi!m  and  «»'^ence  in 
effort  was  also  observed.  Varies  Ideas  are JSJa^  nterac.on  for  amount  of  invested  mental 
data,  including  ,he  effects  of  ?pTr role ^cSSf^r*  f°McSi  r0SuUs  a»d  su'vcy 
Implications  for  future  lr^a^£^^£^™«"  !*««»•* 
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Introduction 


Films  and  videotaped  instructional  programs  serve  ma  ,y  purposes  in  junior  and 

Publ™ Sasting  1984).  Is  Jnumber  of  preexisting  science g  become 
Sable  in  the  more  affordable  VHSformat,  many  *^™*2$3? 
integrate  a  wider  variety  of  videotaped  programs  into  their  curriculum. 

HBMtiMt 


1986). 


The  effects  of  manipulating  an  important  production  variable  on  learning  from  an 

mmmm^ 

OSC)  relationship  in  a  junior  high  school-level  science  film  had  an  effect  on 
achievement  amount  of  invested  mental  effort  (A1ME),  and  attitude. 

The  different  narration  presentation  formats  and  narrator-OSC  rela tionships  were 

^riteSringLl  carrying  out  an  experiment  to  determine  the  effects  of  different 
colors  of  light  on  plant  growth. 

To  create  the  five  experimental  versions,  the  same  general  text  of  *e™»»ton  was 

narration  text  that  changed  for  each  version: 

First  ocrson  voice:       "When  I  select  a  procedure  for  measuring  a  factor  in  my 
First  person  voice.          cxpcrimpRi/  h£c  hcight  and  leaf  count,  I  arn  making  an 

operational  definition." 

c  nA  nnr«,n  voir.-  "When  vou  select  a  procedure  for  measuring  a  factor  in  your 
Second  person  voice.     WJjMjgm^  ^  P  ^  ^  yQU  grc  makmg  an 

operational  definition." 

TWrH  n,»rson  voia.-  "When  Sarah  selects  a  procedure  for  measuring  a  factor  in  her 
1  h.rd  person  vo.ee.       VVhcn^^  ^  ^  ^  ^  ^  ^  .g  maWng  gn 

operational  definition." 


f  ;  o  ' 
O  'i 
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I^™^!***0™^  between  the  narrator  and  OSC  as  well  as  the  different 


narration  presentation  formats  can  be  summarized  as  follows: 


Film  Version: 
Narrator  Agf/y^ff 
Teen  Narrator 
First  Person  (TN1P) 

Teen  Narrator 
Second  Person  (TN2P) 

Teen  Narrator 
Third  Person  (TN3P) 

Adult  Narrator 
Second  Person  (AN2P) 

Adult  Narrator 
Third  Person  (AN3P) 


Narrator-OSC 
Relationship 

Same 

Probably  the  same 
Different 

Probably  different 
Different 


Narration 

Presentation  Form^ 
Story 

Directions 
Story 

Directions 
Story 


response  itemsKuring  thl ^  r^affl' 3^^/°  Ch°iCC  and  instructed 

information  and  procedure IS  JJt  LOmPrchension  of  general  and  specific 


Method 

Subjects 

located  in  a  large  southwest  metnopol  tan  area       haH      werc/rom      same  school  district 
science  is  a  requirement  fo aTl 7$n£c "stSoS,  S?  T     demoSraPhic  Pr°«>cs.  Since 
population  fo?each  school  was  J^X^^*"  ma'nslrcamcd  ™  grade 

Procedure 

immediately  following  the  film  andS  rlZ 8  thC  ,n'ormatlon  would  be  administered 
points.  The  program  Sass^m^^^^™  thcP°s,,cst  would  Degraded  for 
Immediately  ^iij  K^S'^f,  m(mit°r  at  fhc  front  (,f  *ach  classroom, 

posttest  covering  the  informadon  present         81VC"  an  a"ifUdC  surv<*  fo,,(nwd  bV  « 

Criterion  Measures 

th-ecT^  and  9  «™™ed  response  questions  covering 

comprehension  (reliab£ty^9 )   Al°m^orma lio^'festf^1^0     ° T°S  PTCS?ntcd'  ™*  information^ 
the  text  of  the  narration.  'ntormal.on  tested  was  presented  visually  and/or  given  in 

P^^™?SS£t  anKuT*  T iUrin8  in,CrCSt'  fcclin8s  of  evidence  .oward 
effort.  TheSse  moll STcsto  ""g  ^  '""T!  of  ,n Vcs,cd  ™S 

S  wcre  dcr,v«l  from  the  questions  used  by  Salomon  (1983, 
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1984)  to  measure  self-reported  invested  mental  effort  and  included  items  such  as  I  tried  hard  to 
learn  from  this  video,"  and  "Throughout  the  video,  I  concentrated  hard  on  what  was  said by  the 
narrator."  Students  responded  to  all  survey  items  using  a  four  point  Likcrt-type  scale  with  4 
representing  "strongly  agree"  and  1  representing  "strongly  disagree.   Responses  to ^cach  of  the 
four  mental  effort  questions  were  added  together  to  provide  a  maximum  16  point  score  or  self- 
reported  amount  of  invested  mental  effort.  A  reliability  test  on  these  four  questions  yielded  a 
Cronbach  alpha  of  .66. 

Dcsien  and  Data  Analysis  .  ;„kw 

Tnis  study  utilized  a  posttest-only  control  group  design.  The  independent  variables 

included: 

1.  Adult  versus  teen  narrator 

2.  "Story"  versus  "directions"  narration  presentation  format 

3.  Different  narrator-OSC  relationships 

4.  Viewer  gender 

The  dependent  variables  consisted  of  posttcst  performance,  amount  of  invested  mental  effort, 
interest,  and  degree  of  confidence  in  designing  experiments. 

Analysis  of  variance  tests  using  a  5  (version)  x  2  (gender)  factorial  design  with  subjects 
nested  within  classes  nested  within  versions  was  used  to  analyze  the  data  among  groups  for  nil 
dependent  measures  (N=20  for  statistical  purposes).  Single  degree  of  freedom  contrasts  were 
used  to  determine  if  differences  existed  between  adult  and  teen  versions  as  well  as  first,  second, 
and  third  person  versions.  Pearson's  r  test  of  correlation  was  used  to  analyze  the  relationship 
between  posttcst  performance  and  amount  of  invested  mental  effort,  interest,  and  confidence  in 
performing  experiments. 

Results 

Significant  findings  for  each  independent  variable  follow: 
Narrator  "Age" 

ANOVA  results  indicatea  significant  main  effect  for  the  5  versions  F(4,15)=4 .45, 
p<.05.  The  means  for  posttcst  scores  by  gender  arc  reported  in  Table  1.  ANUVA 
sinelc  decree  of  freedom  contrasts  for  posttcst  achievement  scores  indicate  a  significant 
overall  difference  between  the  adult  versions  (M=H-5D  and  teen  narrator  versions  only 
when  the  teen  narrator  first  person  version  is  included  in  the  teen  narrator  group 
(M=12.73),  F(1,15)=7.10,  p<.05. 


Please  insert  Table  1  about  here. 


Comparing  posttcst  scores  between  subjects  viewing  the  adult  and  teen  narrator 
versions  excluding  the  teen  narrator  first  person  version  (a  crossed  design)  did  not  yield 
significant  differences.  Comparing  posttcst  scores  for  subjects  viewing  the  second 
person  adult  and  teen  narrator  versions  did  not  yield  significant  differences  cither. 
When  posttcst  scores  for  the  third  person  versions  were  compared,  the  students  viewing 
the  tcennarrator  version  (M=13.49)  scored  significantly  higher  than  students  viewing 
the  adult  narrator  version  (M=11.69).  Figure  1  illustrates  posttcst  differences  between 
the  adult  and  teen  narrator  second  and  third  person  versions. 


Please  insert  Figure  1  about  here. 
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for  A^EVArnnSndiCatfJ  y icant  8cndcr  by  ™rrator  "age"  interaction  (n<.001) 

^SrSi^^^^^0!"  C°nSiSted  °f  high" lCCn  narra,or  ^riiio' 
AiMt  scores  (M=11.46)  than  the  adult  narrator  version  AIME  scores  (M=10  82)  ronnrieH 

by  females  and  lower  teen  narrator  version  AIME  scores  (M=1(H2  than  adult  »,KS 
version  AIME  scores  (M=11.10)  reported  by  males $0, 5^28 S^DM  {ta^* 
llustrates  this  significant  interaction.  It  is  interesting  to  note  that  all  C'qucs  > ns  used 
to  determine ;  self-reported  amount  of  invested  mental  effort  displayed I  the ?sairo  of 
significant  differences  when  analyzed  separately.  P  yR 


Please  insert  Figure  2  about  here. 


^Sf  results  also  indicated  a  version  main  effect  for  the  survey  item  "If  I  had  the 
SSSfiil'S     .u°  °f Sy  f0r  mC  to  dcsi8n  and  carrX  out  an  experiment  even  more 
SoTln       tllan  thC  f'ant  exPcrin«nt-"  Contrasts  revealed  the  sub Jts  who  viewod 
iwe  ^  Hn  ,. a,°f  V^ons  rcP°rlcd  «  significantly  higher  level  of  confidence  in  being 
able  to  do  complicated  experiments  (M_=2.41)  than  the  subjects  who  viewed I  the  3? 
narrator  versions  (M_=2.12),  F(1,15)=I  1.87,  p<.01.  viewca  tnc  ndult 

Narration  Presentation  Format 

ANOVA  contrasts  revealed  significant  differences  in  posttest  achievement  scores  for 
S59TSK  U^m^  VnrSi°nS  ^»^>»S  third  persTnTeSr" 
versions (d<  05>  ^iLi?  H?n  35  SCCOnd.Person  vcr«ions  versus  all  the  other 

?r*<™j£~i    I     Y   HS    Pairwlse  comparison  test  revealed  significantly  hiehcr 

££  SKS?  F^T  "  ^  ^  *"  ^  ^  ~  ^ 


Please  insert  Figure  3  about  hero. 


Narration-QSC  Relationship 

Single  degree  of  freedom  contrasts  revealed  no  significant  differences  between 
subjects  viewing  the  first  and  third  person  teen  narrator  versions 

Gender 

«JI?  aVCr1g°  ^°SttCSt  SCOrC  for  fcmalc  S"^18  was  higher  than  the  average  nosttes. 
K? ?5mSS ,  p'JS?'-  ™'  mai"  CffCCf  f°r  ^CndCr  aPProa<*cd  signmcant8  F  ' 

ANOVA  results  also  indicated  a  significant  main  effect  for  gender  and  AIMF  with 
ieUmS£Sh!um!f!  '°r,w  su7lX"?m  "™-         program  was  mtemsUm.-  was 
survey  Item  was  2.43,  indicating  a  highly  significant  difference,  F<U5M3 L91  PJ  01 
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Pearson  s  r  test  for  correlation  indicated  there  ^^f^f^A^^ 
oosttest  performance  and  reported  amount  of  invested  mental  effort,  r=.24,  p<.01,  and 
tetw^S  performance  and  interest,  r=.14,  p<-01  Confidence  in  designing  more 
c^^ated  experiments  and  posttest  performance  correlated  significantly  as  well, 
r=.26,  p<-01. 

The  major  findings  in  this  study  arc  summarized  in  Table  2. 


Please  insert  Table  2  about  here. 


Discussion 


The  most  important  finding  in  this  study  was  the  difference  in  posttest  achievement 
duJS^t^^entationW  Junior  high  students  ^^«<£XS^ 
more  information  when  it  was  presented  in  story  format,  particularly  when  the  narrator 
was  dose  to  their  own  age.  These  results  might  be  explained  in  light  of  the  way  both 
Tudio  and  visual  channels  of  information  are  believed  to  be  processed,  as  well  as  the 

^^s^™^  ^  program  used  in  this 

in  both  the  audio  and  visual  channels.  These  channels  correspond  to  the  verbal  and 
visual  stimuli  represented  in  the  integrated  dual-coding  model  of  ^^J* 
986).  Even  though  the  information  presented  in  all  five  versions  was  essentially  the 
same,  the  visual  images  corresponding  to  the  information  may  have  meant  someth.ng 
different  to  the  viewers  depending  on  the  version. 

In  the  first  and  third  person  versions,  the  narrator  described  what  the  central 
character  was  doing.  The  narrator  was  telling  her  story.  In  the  second  person  versions, 
the  v5  images  may  have  been  perceived  as  examples  for  the  information  presented. 
The  narrator  never  acknowledges  the  OSC,  consequently  the  viewers  may  have 
perceived  her  role  as  simp'y  one  of  providing  examples  for  the  information  ^8 
presented  in  the  narration.  If  this  were  the  case,  the  viewers  may  have  been  perce.vmg 
the  verbal  and  visual  stimuli  more  separately  in  the  second  person  versions  than  he 
other Ver  ioS.  They  may  have  been  more  conscious  of  the  differences  in  informa t  on 
presented  in  the  verbal  and  visual  modes,  and  they  may  have  had  a  more  difficult  time 
processing  both  channels  of  information. 

Baeeett  (1984)  and  others  have  described  the  importance  of  presenting  visual  and  audi  tory 
information  simultaneously  for  the  effective  recall  of  information  presented  in  instructional 
films posTible  that  tte  information  presented  in  the  second  person  versions ;  was  not 
preSs  ed  a^exactly  the  same  time  as  Mother  versions  since  the 

mentallv  switching  between  information  and  examples.  Baggett  and  Ehrenfeucht  (1983) 
determined  hat  although  more  processing  time  is  allowed  when  visual  and  verbal  stimuli  ,n  a 
fflm  aTe  presented  eparately,  the  most  ef&tive  recall  strategy  Is  to  present  the  information 
StenSJ  Th«le  results  may  be  explained  in  light  of  the  integrated  dual-coding  model 
SXc nLmatL  storage  and  recall.  It  is  conceivable  that,  when  verba  and  v.sua 
SmuU  arc  praSd  as  a  story,  the  learner  comprehends  both  channels  of  information  at 
xactly  he  Lme  time,  thus  aiding  in  the  development  of  mental  ™™«?™^™Xr 
and  verbal  stimuli.  This  idea  is  consistent  with  the  findings  of  Mayer  and  Anderson  (1991). 
AUhoueh  he  mechanisms  underlying  the  integrated  dual-coding  model  are  not  clearly 
u^nderstood  itTs Reasonable  to  assume  that  a  difference  in  the  perceived  relationship  between 
"he verSl  and  visuaUtimuli  would  result  in  different  types  and  amounts  of  mentaf  connections 
established. 
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Although  quite  speculative,  it  seems  natural  that  the  ideas  and  ramifications  of 
Faivio  s  integrated  dual-coding  model  approach  should  be  applied  to  instructional  films 
and  instructional  television  since  the  presentation  of  visual  and  verbal  stimuli  are  the 
modus  operandi  for  television  and  film  media. 

wl£gffr  ^teSt  *?°reS L°T  th<?  teen  first  and  third  Person  verslons  as  well  as  reported  higher 
2°S  Zin  ^rfTTg  dimCU'/  e*^™"*  ^ter  viewing  the  teen  naffatoTveS  , 
S ii     exPla,Ped  .by  the  effectiveness  of  same-age  role  modeling  by  the  teen  narrator  Role 
model  research  has  indicated  that  adolescents  in  particular  respond  positively  to  KuJ'cJ peer 
role  models  in  academic  settings.  ^     yv»»™y     me  use  or  pi  cr 

rpJSZS  J6  "arratorand  050  were        fema,e' the  hi§.her  lcve's  of  interest  and  A1ME 
reported  by  female  viewers  are  not  surprising.  Beyard-Tyler  and  Sullivan  (1980)  demonstrated 
that  seventh  grade  students  prefer  stories  in  which  the  protagonist  is  their  same  gender  his 
reasonable  to  assume  that  the  females  in  this  study  preferred  the  female  proTaTS  conver  dv 
Sw^Jf         h3V5  Preferml  3  ma,e  narrator  and/or  OSC-  Evcn  though  reLrch  indicates 
989)  ZSSSS!  H?681 Krrted  *  femaIes at  this grade  level  (Linn  and Hyde, 
iim  m  this  shidy  reported  significantly  higher  levels  of  interest  in  this  scicnce-conS 

^SSXSff£SS!e  potcntial  (OT  using  media  t0  help  promotc  highcr  '-e'scofnttn' 

«JK  s'gnirican»  corrc,aHon  between  the  amount  of  invested  mental  effort  and  information 
recall  and  comprehension  supports  the  work  Salomon  and  others  have  done  to  determine  h  vv 

™£&C  ft  l°  1CaT  fl°m  VaHOUS  mcdia  and  how  much  thcy  actuan^do  learn  Z 
narrator  age  by  gender  for  amount  of  invested  mental  effort  interaction  was  an  unexoocted 

SSSSfSJ;    p058  blc  thu ,he  malcs  fclt  the  CTedibility of  thc  -^^JEJ^n 

hat  of  the  teen  narrator  so  they  concentrated  harder  on  what  was  going  on  in  the  film  and 
ound  the  nformation  easier  to  understand.  In  any  case,  thc  gender  main  e  fecUndTmcracti(»n 

SlSS?     'nVeS  tCd  "u™131  Cffort  ilIuStrate  the  need  t0  look  at  other^producUonfacto  s 
..ivolved  in  connectmg  the  presentation  content  with  the  audience. 

The  results  from  this  study  indicate  that  learning,  amount  of  invested  mental  effort 

n  option  **  SimP,e  chanSes  in  *e  presentation  of  narrated  ' 

e  u hS $1  Slf   r  f ,gh  School-'cvcl  fil™-  Many  questions  were  raised  as  a 

result  of  this  study.  Future  research  in  this  area  might  include  determining  whether 

^o^PZSmtCd  *h  fifSt  °r  third  P™0"  V0icc  ?ea,,y  is  in,crPrc^  a*  mor5  of  a 
2(fo7  S(  scc°nd  Pf rson  voice.  It  would  also  be  interesting  to  see  how  the 

m im™  Ln/ioH  f  ma'C  na^a«r  3nd  ^  wcrc  uscd  in  a  sim'lar  film-  '« ™uld  also 
prove  important  to  determine  if  different  presentation  formats  affect  the  recall  and 

comprehension  of  information  presented  only  in  the  visual  channel. 

Since  the  use  of  video-taped  films  are  widespread  in  science  education,  more  research 
dealing  with  film  and  v  dec  information  presentation  variables  is  needed   T [  ensure 

tdttt  t^J**  ?  Cffectivcncss'  c((0Tts  ^ould  be  made  to  apply  current  research 
findings  to  the  development  of  new  instructional  films. 
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Table  1. 


Mean  Postt^t  Scor^  fry  f^n^r 

N*imQL  Females  Mp|f. 


Teen 

First  Person 


Third  Person 
Adult 


Total 


Total 


13.56  (2.61)       12.98  (3.43)  13.27(3.04) 
Second  Person  10.98(3.60)       11.98  (3.74)       11.42  (3.67) 

14.25  (3.24)       12.71  (3.95)       13.49  (3.66) 


Second  Person  11.95(3.08)       11.23(4.02)       11.53  (3.66) 

Third  Person  12.79(3.46)       10.64(3.73)  11.69(3.74) 

12.57(3.42)       11.84(3.85)  12.20(3.66) 


Noje,  Total  N=441  with  35-53  per  cell. 
Standard  Deviations  in  parentheses. 
Maximum  posttest  score  =  20. 
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Table  2. 

Summary  of  Results 


Independent 
Variable 


Narrator  "Age" 


Results 


Students  viewing  the  third  person  versions  recalled  and  comprehended 
more  content  when  the  narrator  was  their  own  age. 

Students  were  more  confident  they  could  do  more  difficult  experiments 
on  their  own  when  the  narrator  was  their  own  age. 

Females  reported  higher  AIME  for  the  teen  narrator  versions,  while 
males  reported  higher  AIME  for  the  adult  narrator  versions. 


Narration 

Presentation 

Format 


Gender  Differences 


Students  recalled  and  comprehended  more  content 
when  the  narration  was  presented  in  "story" 
format  (first  or  third  person). 

Females  reported  higher  AIME  than  males. 

Females  were  more  interested  than  the  males  in  the  film. 

Females  were  more  interested  than  males  in  doing  experiments  in  class 
similar  to  the  one  demonstrated  in  the  film. 


^  li  U 
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Figure  1-  Mean  Posttest  Scores  by  Narrator  "Age" 
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Pigm&2.  Narrator  "Age"  by  Gender  InteracHon  for  AIME 
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-  Female  A1ME 


•  Male  AIME 
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Fifiur£3.  Mean  Posttest  scores  by  Narrator  Presentation  Format. 
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In.truct.on  Material,  and  Audio-Tape  Material.  inT.c«cT„g  Chine.e 

audio  ^Tlf^T1**^  hM  tIready  indic^*1  "*>»  computer-bawd  and 
?aw££?  ftSSSTf  T  in<L6!M  ,tud6nU'  P*rfo™*»ce  in  learning  rSreSf 
iSto^cete^ 

CnTse3^^^ 
&e^ 

elTeTta^Set^ 

included  item,  concerning  the  sfuSenS taftK 
language,  and  experience  with  Mandarin  Chines  wa.  riven  to  .5  th? 
participant.  All  subject,  indicated  that  they  haJalSt  nl  in 
learnmg  a  foreign  language  but  not  .penally  ChYnesT 

...    The  textbook  and  the  audio  tape  used  in  the  class  were  nrenared  hv  th» 

SSSXTh  ^  Pri°rity  WM  *~  * 
interZZ  TZ  i  communication,  entertainment,  social 

Wstorv  ?»«? u    "  8tUdied  80,1,6  inf°™ation  on  China's  culture  and 
2£2 "~  Ph°net,C8  811(1  grammar  Were  dea,t        in  the  Audio 

MadntoIheH3t^"ba8ed  "Mt£?1  WM  deve,0ped  b*  the  *****  using  the 

Artox^Sria?  T^S  ?C  ,e88°n8  th8t  thC  aUth°r  Created  on 
h^twp«nf°  nu  Pnonty  was  given  to  the  recognition  of  the  difference 

In  order  to  eliminate  the  possibilities  of  unfamiliaritv  with  th* 
n,i„«   p   u  *  ^*tS  W8re  deveI°P«d  by  the  Chinese  instructor  who  came  from 

eight  weeks.  Subjects  performance  on  each  part  of  the  test  were  reported  to 
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the  author  weekly.  The  mean  score  of  each  subject  was  compared  to  the  others  to 
find  out  the  difference  between  treatments. 

The  result*  illustrated  that  students  performed  better  in  the  weeks  in  which 
computer-based  materials  were  used.  The  results  also  indicated  that  in  some  skill 
areas  the  two  instructional  tools  did  not  show  a  noticeable  difference  in  helping 
subjects  to  learn  Chinese.  For  example,  in  the  listening  comprehension  part,  the 
performance  of  the  subjects  did  not  show  an  obvious  difference  when  compared. 
Results  also  indicated  that  subjects  performed  the  best  on  the  translation  part  while 
they  were  using  the  CBI  material.  It  may  have  been  because  subjects  had  the  chance 
to  actively  practice  their  reading,  writing,  and  recognizing  skills  directly  from  the 
CBI  materials.  In  the  character  writing  part,  results  showed  that  students 
performance  was  better  on  the  weeks  in  which  CBI  was  used.  Also,  according  to  the 
subjects'  self-evaluation,  the  CBI  was  rated  higher  than  the  audio  tape  in  delivering 
the  lessons.  Subjects  also  reported  that  they  needed  less  time  on  mastering  the 
lessons  while  the  CBI  was  used. 

Although  the  results  of  the  study  indicated  that  using  CBI  to  help  practice 
Chinese  was  better  than  the  use  of  audio  tape,  it  was  limited  in  recognition  of 
students'  utterance  in  order  to  help  them  practice  the  speaking  skill.  Furthermore, 
the  study  had  demonstrated  that  it  is  possible  to  use  CBI  materials  to  teach  non- 
alphabetical  languages.  It  is  recommenced  that  further  study  to  compare  the  two 
different  instructional  tools  is  appropriate.  Future  studies  should  explore  the 
capability  of  integrating  many  instructional  methods  in  the  development  of  CBI 
instructional  material  for  teaching  a  non-alphabetic  language  like  Chinese, 
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Abstract 

This  study  examined  the  effects  of  instructional  control  and  aptitude  composition  on 
performance,  interaction,  and  attitudes  during  a  computer-based  cooperative  science  lesson. 
A  sample  of  152  fifth  and  sixth  grade  students  completed  the  instruction  either  under  the 
learner  control  (LC)  or  the  program  control  (PC)  treatment.  In  the  LC  treatment,  students 
exercised  control  over  the  amount,  review,  and  sequence  of  instruction.  Students  in  the  PC 
treatment  followed  a  predetermined  instructional  path.  At  the  end  of  the  lesson,  all  students 
completed  a  posttest  and  an  attitude  questionnaire.  They  also  indicated  their  confidence  by 
predicting  the  score  on  the  posttest.  An  identical  test  was  adrninistered  two  weeks  later  for 
measuring  retention.  Time  for  each  group  and  interaction  in  representative  groups  were 
also  recorded.  Results  suggested  that  both  heterogeneous  grouping  and  learner  control  had 
significant  effects  on  learning,  time  on  task,  verbal  interaction,  and  attitudes.  Working  in 
heterogeneous  groups  increased  confidence  more  than  working  in  homogeneous  groups. 
High-ability  students  outperformed  low-ability  students  on  all  dependent  variables,  except 
attitudes.  The  implications  for  designing  group-based  microcomputer  software  and  future 
research  are  discussed. 
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The  Effects  of  Learner  Control  and  Group  Composition 
in  Computer-Based  Cooperative  Learning 
It  is  often  argued  that  microcomputers  are  capable  of  delivering  effective  instruction 
by  adjusting  the  content  to  cognitive  and  affective  needs  of  each  student  This  capability 
had  considerable  effects  on  instructional  designers  by  encouraging,  in  some  cases  forcing, 
themtodevelopleamingriterials  which  ««^»Wh«dtt^€f--fc-. 
Efforts  toward  integrating  the  power  of  computers  with  the  popularity  of  individualized 
instruction  have  long  dominated  the  educational  software  market. 

The  evidence  from  research  shows  that  individualized  instruction  may  be  effective 
under  certain  conditions.  Using  famUiar  contexts  for  learning  consistently  yielded  better 
performance  and  attitudes  (Ross  &  Morrison,  1989).  Adapting  the  lesson  sequence  or  the 
difficulty  of  practice  items  to  individual  differences  promoted  higher  student  achievement 
(Tennyson,  1981).  Tailoring  the  instruction  to  progress  and  needs  of  each  student  resulted 
in  better  learning  efficiency  (Wsuig  &  Walberg,  1983). 

However,  there  is  also  evidence  showing  that  individualized  instruction  may  not 
always  be  desirable.  Although  educators  in  public  schools  often  assume  that  computer- 
based  instruction  must  incorporate  an  individualistic  paradigm  of  learning,  researchers  have 
raised  serious  questions  regarding  the  effectiveness  of  individualized  instruction  (Hooper, 
1992;  Johnson  &  Johnson,  1986;  Mevarech,  Stem,  Levita,  1987). 

It  appears  that  even  the  most  successful  form  of  individualized  instruction  may  have 
some  problems.  First,  the  high  cost  and  complexity  of  designing  adaptive  instruction  that 
caters  to  a  wide  variety  of  learning  needs  may  impose  severe  limitations  on  the  potential  for 
delivering  individualized  instruction.  Practical  constraints  become  particularly  challenging 
when  students  need  separate  workstations  for  completing  instruction  (Hooper  &  Hannafin, 
1991).  Second,  individualized  instruction  may  reduce  student  experiences  to  the  resources 
provided  only  by  the  learning  environment.  This  approach  denies  the  benefits  of  peer 
interaction  which  can  be  highly  useful  in  many  learning  situations  such  as  those  involving 
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piosocial  behavior,  group-to-individual  transfer,  and  creative  skills  (King,  1989;  Webb, 
1  v82).  Finally,  working  alone  for  extended  periods  may  create  isolated  situations  in  which 
students  are  bored,  anxious,  and  frustrated.  These  feelings  may  have  deteriorating  effects 
on  students'  achievement  motivation  (Johnson  &  Johnson,  1986). 

The  limitations  of  individualized  instruction  have  forced  educators  to  develop  more 
effective  strategies.  Cooperative  learning  has  been  suggested  as  a  powerful  alternative  to 
individualized  instruction  (Johnson  &  Johnson,  1989;  Kagan,  1992;  Sharan,  1980;  Slavin, 
1983).  However,  there  are  a  number  of  cooperative  learning  methods.  What  characterizes 
these  methods  is  that  students  work  together  in  small  groups  to  accomplish  a  common  goal. 
Such  a  collaboration  usually  involves  positive  interdependence  promoting  mutuality  among 
students,  individual  accountability  as  a  measure  of  personal  responsibility,  peer  interaction 
generating  instructional  support,  social  skills  necessary  for  working  effectively  with  others, 
and  group  processing  as  a  way  of  improving  team  performance. 

A  large  body  of  research  suggests  that  cooperative  learning  methods  may  be  more 
effective  than  competitive  and  individualistic  experiences  in  promoting  desired  educational 
outcomes.  For  example,  a  meta-analysis  that  examined  226  studies  comparing  the  impact 
of  cooperative  and  individual  structures  on  student  performance  yielded  an  average  effect 
size  of  .63  in  favor  of  cooperation.  Similar  results  were  reported  for  die  comparison  of 
cooperative  and  competitive  methods  (Johnson  &  Johnson,  1989).  Most  of  these  studies, 
however,  were  conducted  in  regular  classrooms  with  limited  use  of  emerging  interactive 
technologies  such  as  computers  and  videodiscs.  The  effectiveness  of  technology-based 
cooperative  learning  environments  have  been  examined  only  in  recent  years.  The  results 
from  several  integrative  reviews  suggested  that  cooperative  learning  which  incorporate 
new  technologies  improved  both  cognitive  and  affective  outcomes  (Rysavy  &  Sales,  1991; 
Webb,  198£). 

Although  it  is  not  known  what  exactly  mediates  the  effectiveness  of  cooperative 

learning,  group  composition  is  frequently  mentioned  as  a  potential  factor  producing  better 
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achievement  and  attitudes.  Cooperative  learning  usually  involves  heterogeneous  grouping; 
that  is,  groups  are  formed  to  include  students  from  discrepant  aptitudes,  gender,  social 
status,  and  ethnic  backgrounds.  Despite  this  common  use,  researchers  are  not  in  complete 
agreement  on  the  efficacy  of  heterogeneous  grouping. 

Supporters  of  heterogeneous  grouping  claim  that  all  students  benefit  from  working 
with  partners  of  different  abilities.  It  is  argued  that  high  ability  members  of  the  group  may 
take  an  active  role  in  providing  support  and  guidance  to  their  low  ability  partners.  These 
activities  often  involve  the  use  of  generative  strategies  that  makes  the  learning  experience 
more  meaningful  (Wittrock,  1990).  Heterogeneous  grouping  may  also  provide  beneficial 
opportunities  for  low  ability  students.  As  a  consequence  of  receiving  personalized  help  and 
academic  support  from  more  able  partners  in  the  group,  low  ability  students  may  improve 
their  personal  expectations  for  competence  and  invest  more  effort  toward  accomphshing  the 
group  goal  (Cohen,  Lotan,  CatanzariteN**)).  A  number  of  studies  demonstrated  that  all 
students  benefited  from  working  in  heterogeneous  groups  compared  to  working  alone 
(Dalton,  Hannafin,  &  Hooper,  1989;  Gabbert,  Johnson,  &  Johnson,  1986;  Hooper,  1992; 
Johnson,  Skon,  &  Johnson,  1980;  Simsek  &  Hooper,  1992;  Yager,  Johnson,  Johnson,  & 
Snider,  1986).  There  is  also  evidence  showing  that  heterogeneous  grouping,  compared  to 
homogeneous  one,  is  particularly  effective  for  less  able  students  while  it  is  not  detrimental 
for  their  high  ability  partners  (Hooper  &  Hannafin,  1991;  Simsek  &  Tsai,  1992 ). 

Critics  of  heterogeneous  grouping  claim  that  it  promotes  personal  gains  for  some 
students  at  the  expense  of  others,  but  these  researchers  are  divided  as  to  who  benefits  most 
from  heterogeneous  group  work.  After  reviewing  some  studies,  Hill  (1982)  concluded  that 
the  achievement  of  high  ability  students  was  hindered  by  medium  and  low  ability  partners. 
Beane  and  Lemke  (1972),  on  the  other  hand,  found  that  heterogeneous  grouping  helped 
high  ability  students  more  than  did  other  members  of  the  group.  Similar  questions  were 
raised  for  the  achievement  of  low  ability  students.  Slavin  (1983)  argued  that  heterogeneous 
grouping  may  offer  few  opportunities  for  low  ability  students  when  they  are  simply  given 
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the  correct  answers  without  explaining  the  solution  process  behind  answers.  Peterson  and 
Janicki  (1979)  supported  that  high  ability  students  performed  better  and  had  more  positive 
attitudes  in  small  groups,  whereas  low  ability  students  gained  more  from  traditional  whole 
class  instruction.  Goldman  (1965)  provided  contrasting  evidence  indicating  that  students 
working  with  partners  of  higher  abilities  performed  better  than  those  working  with  partners 
of  similar  or  lower  abilities.  Researchers  also  suggested  that  average  students  are  often 
neglected  in  heterogeneous  groups.  Swing  and  Peterson  (1982)  found  that  peer  interaction 
in  mixed  groups  facilitated  the  achievement  of  high  and  low  ability  students,  but  did  not 
affect  the  achievement  of  medium  ability  students.  Webb  (1982)  reported  similar  findings 
suggesting  that  average  students  in  homogeneous  groups  showed  better  achievement  and 
received  more  explanations  than  average  students  in  heterogeneous  groups.  It  appears  that 
even  the  critics  of  heterogeneous  ability  grouping  agree  on  the  effectiveness  of  cooperation 
in  small  groups,  but  they  do  not  have  a  clear  consensus  as  to  what  types  of  students  benefit 
most  from  this  experience. 

The  ^conclusiveness  of  research  findings  on  cooperative  group  composition  forces 
educators  to  develop  better  intervention  strategies  that  will  enhance  the  performance  of  all 
students  working  in  small  groups.  One  strategy  for  providing  more  effective  instruction  is 
to  give  students  an  opportunity  to  have  control  over  their  own  learning.  Exercising  control 
based  upon  group  decisions  may  help  students  develop  better  metacognitive  skills  needed 
for  successful  learning. 

The  concept  of  learner  control  is  intuitively  appealing  because  students  differ  in 
aptitudes,  interests,  motivation,  learning  styles,  and  personality  variables  (Carrier,  1984). 
The  results  from  a  number  of  experimental  studies  suggest  that  there  are  certain  situations 
in  which  exercising  control  is  more  effective  than  following  a  fixed  instructional  sequence. 
When  students  were  allowed  to  adjust  the  familiarity  of  content  to  their  own  backgrounds, 
for  example,  they  performed  significantly  better  on  both  achievement  and  attitude  measures 
(Ross,  1984).  Similarly,  providing  students  with  advice  about  their  academic  performance 
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and  learning  needs  has  been  found  to  be  more  effective  than  imposing  a  fixed  instructional 
sequence  rr  complete  learner  control  (Tennyson  &  Rothen,  1979).  Researchers  argue  that 
guided  learner  control  that  is  contingent  upon  students'  ongoing  performance  combines  the 
most  desirable  features  of  adaptive  instructional  support  and  personal  decision  making. 

Studies  also  demonstrate  that  practicing  control  does  not  always  promote  desired 
educational  outcomes.  Given  total  learner  control,  students  usually  terminate  the  lesson 
prematurely  or  they  cannot  make  appropriate  decisions  regarding  the  instruction  they  need 
to  complete  (Carrier,  1984;  Kinzie.  1990;  Snow,  1980;  Tennyson,  1981).  It  has  been 
suggested  that  particularly  younger  and  less  able  students  benefit  very  lime  from  exercising 
control,  compared  to  mature  and  more  able  students  (Hannafin,  1984;  Steinberg,  1989). 
An  explanation  for  these  results  may  be  that  low  ability  students  usually  lack  metacognitive 
skills  that  are  critical  in  evaluating  the  situation  and  producing  effective  solutions.  Having 
failed  frequendy  on  academic  tasks,  these  students  may  be  more  comfortable  with  external 
instructional  support  (Simsek  &  Sales,  1992). 

Although  there  seems  to  be  a  strong  agreement  in  the  uterature,  only  several  studies 
examined  the  relationship  between  student  ability  and  instructional  control.  Ross  and 
Rakow  (1981)  conducted  a  study  in  which  the  number  of  supporting  examples  was  adapted 
to  pretest  scores  through  program  control,  selected  through  learner  control,  kept  constant, 
or  nonadaptive  content  was  presented  through  traditional  lecture.  Results  demonstrated  that 
program  control  of  instructional  support  promoted  highest  mathematics  achievement  on 
both  immediate  and  delayed  posttests  while  learner  control  resulted  in  lowest  achievement. 
Nonadaptive  support  and  traditional  lecture  treatments  produced  medium-range  outcomes. 
The  results  also  suggested  that  program  control  was  more  effective  for  low  ability  students. 
Fry  ( 1972)  found  similar  findings  indicating  that  high  ability  students  learned  significandy 
more  under  a  high  degree  of  learner  control.  Gay  (1986)  reported  that  students  with  higher 
prior  knowledge  performed  equally  well  under  both  program  control  and  learner  control 
treatments,  but  students  with  low  prior  understanding  scored  less  under  learner  control  as  a 
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result  of  poor  sequencing  decisions.  It  is  important  to  note  that  all  of  these  studies  were 
conducted  in  college  environments  with  relatively  mature  participants.  Results  appear  to  be 
different  with  younger  students.  For  example,  Klein  and  Keller  (1990)  found  that  type  of 
instructional  control  did  not  affect  performance  and  confidence  of  seventh  grade  students, 
whereas  ability  and  locus  of  control  improved  the  same  outcomes.  More  research  needs  to 
be  conducted  in  elementary  classrooms  for  obtaining  conclusive  results. 

It  is  also  true  that  most  of  the  conclusions  about  learner  control  have  been  derived 
from  studies  on  individualized  instruction.  The  effects  of  exercising  learner  control  in 
cooperative  groups  are  not  known  yet.  It  is  crucial  for  instructional  designers  to  recognize 
that  research  findings  from  individualized  instruction  may  not  necessarily  be  generalizable 
to  small  group  learning.  When  high  ability  students  dominate  the  group,  for  example,  low 
ability  students  may  suffer  from  completing  too  little  or  difficult  instruction.  When  less 
able  students  dominate,  on  the  other  hand,  more  able  students  may  suffer  from  completing 
too  much  instruction  or  being  bored  (Simsek  &  Hooper,  1992). 

The  basic  question  still  remains:  What  will  be  the  effects  of  exercising  control  on 
performance,  interaction,  and  attitudes  of  students  working  together  in  homogeneous  and 
heterogeneous  ability  groups  during  a  computer-based  cooperative  science  lesson?  Further 
research  is  needed  to  answer  this  question.  The  purpose  of  the  present  study  is  to  extend 
research  findings  by  determining  whether  exercising  control  over  the  amount,  sequence, 
and  review  of  instruction  in  cooperative  groups  would  be  more  effective  than  following  a 
fixed  instructional  path. 

Method 

5ubjfiClS 

A  sample  of  152  fifth  and  sixth  grade  students  from  a  suburban  school  district  in 
Minnesota  participated  in  the  study.  Of  this  total  number,  84  (55%)  were  fifth  graders  and 
68  (45%)  were  sixth  graders.  The  gender  distribution  was  73  (48%)  males  and  79  (52%) 
females.  The  sample  comprised  primarily  of  middle-class,  Anglo-American  students  with  a 
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small  percentage  of  other  ethnic  backgrounds.  Approximately  equal  number  of  subjects 

were  randomly  assigned  to  treatment  conditions. 

Materials 

Lfisson  Content.  Students  completed  a  tutorial  lesson  on  solar  energy.  TheCBI 
lesson  was  originally  developed  by  Kinzie,  Sullivan,  and  Berdel  (1938)  for  individualized 
instruction.  Modified  versions  of  the  lesson  for  both  individuals  and  cooperative  groups 
were  prepared  and  validated  in  another  study  (Simsek  &  Sales,  1992).  Based  upon  the 
results  of  these  studies  and  according  to  design  principles  described  by  Dick  and  Carey 
(1985),  a  formative  evaluation  procedure  was  conducted.  Final  version  of  the  computer 
lesson  contained  five  major  segments:  (a)  solar  power,  (b)  active  solar  heating;  (c)  passive 
solar  heating;  (d)  solar  cells;  (e)  future  of  solar  energy.  In  addition  to  these  instructional 
segments,  the  lesson  included  an  introduction  explaining  the  rules  of  cooperation  and  a 
closing  with  comments  about  the  performance  of  students.  Relative  to  the  control  strategy 
embedded  in  the  lesson,  students  received  instruction  that  varied  by  amount,  review,  and 
sequence. 

Ability  Test.  Students  were  classified  as  high  and  low  ability  according  to  their 
performance  on  the  Iowa  Test  of  Basic  Skills.  Students  with  combined  scores  at  or  above 
the  60th  percentile  were  defined  as  high  ability,  while  low  ability  students  were  defined  as 
those  with  the  combine*  scores  at  or  below  the  40th  percentile.  As  a  way  of  reducing  the 
classification  error,  students  with  combined  scores  between  the  60th  and  40th  percentiles 
were  not  included  in  the  study.  The  KR-20  reliability  for  this  standardized  ability  test  for 
both  5th  and  6th  grade  students  was  .91. 

Easiest.  Students  completed  an  individual  posttest  both  at  the  end  of  the  instruction 
and  approximately  two  weeks  after  the  instruction.  The  test  included  40  multiple-choice 
questions.  Half  of  the  questions  consisted  of  recall  items  whereas  the  other  half  contained 
comprehension  items.  The  KR-20  reliability  for  this  posttest  was  .83  with  an  average  item 
difficulty  of  .67.  A  typical  recall  item  on  the  test  was:  "What  is  the  basic  source  of  energy 
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that  all  living  creatures  use?  (a)  Sunshine;  (b)  Electricity;  (c)  Gasoline;  (d)  Fossil  fuels".  A 
typical  comprehension  item  was:  "Which  of  the  following  is  more  likely  in  the  twenty-first 
century?  (a)  Fossil  fuels  will  no  longer  be  used;  (b)  Solar  energy  will  be  the  alternative  to 
nuclear  power,  (c)  Most  of  the  old  buildings  will  be  using  passive  solar  heating;  (d)  Water 
will  be  the  main  source  of  energy". 

Attitude  Owfrfirionnairc-  Following  the  immediate  posttest,  students  responded  to  a 
Likert-type  questionnaire.  This  instrument  contained  30  items,  divided  equally  among  the 
categories  of  attitudes  toward  the  delivery  system,  subject  matter,  and  team  work.  Possible 
responses  ranged  from  "Strongly  Agree  (5)"  to  "Strongly  Disagree  (1)".  The  Cronbach's 
Coefficient  Alpha  reliability  for  the  questionnaire  was  .91.  A  typical  item  for  each  category 
was:  "I  enjoyed  working  with  the  computer"  (delivery  system);  "I  would  like  to  learn  more 
about  solar  energy"  (subject  matter);  and  "I  feel  more  comfortable  working  in  a  small  group 
than  working  alone"  (team  work). 

Intwraction  Scale  Peer  interaction  in  twenty  representative  groups  was  videotaped 
and  analyzed  according  to  categories  of  the  Interaction  Scale.  This  scale  contained  ten 
categories:  (a)  reading  information;  (b)  asking  questions;  (c)  giving  answers;  (d)  seeking 
clarification;  (e)  providing  explanations;  (f)  making  comments;  (g)  energizing  the  group; 
(h)  suggesting  directions;  (i)  entering  group  response;  (j)  demonstrating  off-task  behavior. 
The  coding  of  peer  interaction  in  groups  was  performed  by  two  well-trained  observers  who 
had  expertise  on  cooperative  learning  and  computer-based  instruction.  Several  videotapes 
were  coded  by  both  observers  for  calculating  reliability.  Analysis  of  group  interaction  also 
included  qualitative  data  regarding  the  nature  and  degree  of  student  discourse. 
Training  n^ryfirfi 

Two  observers  were  trained  about  analyzing  interaction  in  small  groups.  Training 
comprised  of  several  stages.  First,  the  experimental  procedures  were  explained  and  each  of 
the  categories  in  the  Interaction  Scale  was  discussed.  Second,  the  observers  watched  a 
videotape  that  showed  two  students  working  together  on  the  same  computer.  This  activity 
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allowed  tl*  observers  to  pay  close  attention  to  behaviors  involving  group  work  and  discuss 
them  in  detail.  Third,  the  observers  coded  peer  interaction  in  a  sample  group  by  using  the 
interaction  Scale  described  above.  Finally,  independent  codings  of  the  two  observers  were 
compared.  The  percentage  of  agreements  between  the  observers  was  .98. 

^pprative  Skills  Training 

Students  in  the  participating  school  had  been  trained  extensively  about  collaborative 
skills.  Cooperative  learning  is  considered  to  be  an  integral  part  of  the  school  curriculum. 
In  addition  to  previous  training,  all  possible  subjects  attended  a  one-hour  training  program 
designed  specifically  for  them.  The  main  purpose  behind  this  activity  was  to  encourage 
students  to  employ  specific  strategies  involved  in  this  study.  Participants  completed  several 
exercises  emphasizing  the  basic  principles  of  cooperative  learning. 

The  first  exercise,  called  "Magic  Triangle"  (Johnson,  Johnson,  &  Holubec,  1991), 
required  students  to  find  the  maximum  number  of  embedded  triangles  in  a  big  one.  Half  of 
the  participants  completed  this  exercise  in  cooperative  groups,  while  the  other  half  worked 
alone.  The  result  was  overwhelmingly  better  success  rate  for  those  working  in  groups. 
Therefore,  successful  students  shared  their  answers  with  all  participants  and  explained  the 
strategies  they  employed.  This  helped  students  to  draw  the  conclusion  that  working  with 
others  can  be  more  effective  and  enjoyable  than  working  individually.  Following  this 
activity,  the  discussion  focused  on  specific  behaviors  which  were  helpful  (i.e.  explaining) 
and  not  helpful  (i.e.  teasing)  for  group  success. 

Second,  the  "Broken  Circles"  exercise  (Cohen,  1986)  emphasized  the  importance 
of  positive  interdependence  among  members  of  a  cooperative  group.  Each  group  was 
given  pieces  of  broken  circles.  The  task  was  to  form  at  least  one  complete  circle  for  each 
member.  Once  everyone  in  the  group  had  a  complete  circle,  members  were  to  shuffle  their 
pieces  and  form  new  circles.  They  had  to  exchange  the  pieces  voluntarily  without  being 
asked  for.  Students  were  also  instructed  not  to  talk  and  take  others'  pieces  unless  offered. 
This  game  encouraged  students  to  share  their  resources  in  accomplishing  the  mutual  goal. 


963 


12 

The  third  exercise,  called  "Contractions  and  Punctuations"  (Johnson,  Johnson,  & 
Holubec,  1991),  was  about  correcting  grammar  errors  in  a  letter.  It  provided  students  with 
opportunities  to  practice  cooperative  skills  on  a  school-related  task.  After  this  activity  and 
presentations  of  several  students  demonstrating  correct  answers,  participants  were  asked  to 
discuss  how  well  their  group  did  as  a  team  and  list  three  advantages  and  disadvantages  of 
working  together  with  others  in  a  small  group. 
Procedures 

Based  upon  their  combined  scores  on  a  standardized  ability  test,  students  were 
randomly  assigned  to  homogeneous  and  heterogeneous  groups.  A  homogeneous  group 
included  two  students  of  the  same  ability,  while  a  heterogeneous  group  included  one  high 
and  one  low  ability  student  Prior  to  the  experiment,  all  students  who  obtained  parental 
consent  participated  in  the  training  program 

Students  completed  the  instruction  in  three  consecutive  days,  approximately  thirty 
minutes  a  day.  To  secure  confidentiality,  each  subject  was  assigned  a  code  number.  They 
were  also  given  a  card  showing  their  computer  number.  During  the  first  day,  students 
completed  two  instructional  segments.  At  the  end  of  the  daily  session,  they  discussed  how 
well  their  group  did  and  listed  two  behaviors  that  were  helpful  and  not  helpful  for  their 
success.  They  also  specified  a  behavior  which  they  promised  to  improve  for  the  next  day. 
Students  began  the  second  day  by  discussing  the  important  rules  of  cooperation  as  ;/ell  as 
the  information  they  learned  from  the  first  day.  Then,  they  studied  two  additional  segments 
of  the  lesson.  The  second  day  ended  with  similar  group  processing  activities.  Students 
completed  the  last  instructional  segment  in  the  third  day.  Following  a  brief  wrap-up  time, 
the  immediate  posttest  and  attitude  questionnaire  were  administered.  Two  weeks  later,  r  n 
identical  posttest  was  given  to  measure  retention. 
Treatments 

During  the  instruction,  students  completed  either  the  learner  control  or  program 
control  version  of  the  computer  lesson.  Each  instructional  treatment  varied  by  the  type  and 
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degree  of  control  provided.  Major  differences  between  the  two  versions  of  the  lesson  are 
described  below. 

I  rme.r  Control.  This  treatment  allowed  students  to  exercise  control  over  the 
amount,  review,  and  sequence  of  instruction.  Following  the  introduction,  group  members 
selected  a  segment  from  the  main  menu.  They  had  complete  control  over  the  segment  they 
wanted  to  start  with.  After  reading  and  discussing  the  information  in  each  lesson  segment, 
students  responded  to  embedded  questions.  They  had  an  option  of  deciding  the  number  of 
practice  items  they  wanted  to  see.  Before  they  answered  any  question,  the  computer  told 
them  how  many  items  (ranging  from  4  to  7)  were  available  in  that  particular  segment  The 
whole  lesson  contained  total  of  25  practice  items.  Because  this  was  a  cooperative  lesson, 
group  members  had  to  carefully  examine  the  options  and  reach  a  consensus  before  entering 
a  response. 

Based  on  their  answer,  students  received  immediate  feedback.  When  the  response 
was  correct,  affirmative  feedback  with  a  statement  saying  'That's  correct!"  was  presented. 
When  their  response  was  incorrect,  however,  students  were  presented  a  statement  saying 
"Sorry.  That's  incorrect.  Would  you  like  to  review  the  material  before  you  answer  the 
question  one  more  time?"  Then,  group  members  either  reviewed  the  relevant  segment  or 
responded  to  the  same  question  again.  Review  screens  contained  shorter  information  than 
original  screens  of  the  tutorial.  Following  the  third  incorrect  response,  the  correct  answer 
with  a  brief  explanation  saying  like  "The  correct  answer  is  (a)  because  all  living  creatures 
need  sunlight  to  grow"  was  displayed.  After  each  segment,  the  main  menu  was  presented. 
Students  decided  to  study  another  segment  or  terminate  the  lesson.  Once  they  selected  a 
segment,  they  could  not  exit  until  the  practice  items.  At  the  end  of  the  instruction,  students 
received  an  evaluative  comment  about  their  overall  performance. 

Prn^m  Control.  This  treatment  did  not  allow  students  to  exercise  control.  All 
students  completing  this  version  of  the  lesson  received  identical  instruction.  Students  did 
not  make  fundamental  decisions  or  choices  about  the  instructional  sequence.  They  simply 
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read  the  information  and  answered  all  embedded  questions.  Similar  to  the  learner  control 
treatment,  students  had  to  carefully  examine  the  options  and  reach  a  group  consensus  about 
their  response. 

When  the  answer  to  a  question  was  correct,  students  received  affirmative  feedback 
saying  "That's  correct!"  and  proceeded  to  the  next  question.  When  incorrect,  on  the  other 
hand,  a  review  of  the  relevant  segment  was  automatically  presented  following  a  statement 
saying  "That's  incorrect  Let's  review  the  material  before  you  answer  the  question  again". 
Then,  students  responded  to  the  question  one  more  time.  After  three  incorrect  responses, 
the  correct  answer  with  a  brief  explanation  was  presented.  Upon  completion  of  all  lesson 
segments,  students  received  an  evaluative  comment  about  their  overall  performance. 
Design  anH  nata  Analysis 

The  study  employed  a  2  X  2  X  2  randomized  block  design.  The  first  factor  was 
Grouping  (homogeneous,  heterogeneous),  the  second  Ability  (high,  low),  and  the  third 
Control  (learner,  program).  Dependent  variables  were  achievement,  confidence,  retention, 
attitudes,  time  on  task,  and  interaction.  In  analyzing  research  data,  three-way  ANOVA  and 
Scheffe's  multiple  comparisons  test  were  used.  The  alpha  level  for  testing  differences  was 
set  as  .05,  unless  otherwise  indicated. 

Results 

Achievement 

The  first  dependent  variable  of  the  study  was  student  achievement  This  variable 
was  operationalized  as  the  individual  score  on  the  immediate  posttest.  Means  and  standard 
deviations  for  achievement  scores  are  given  in  Table  1.  The  mean  score  for  heterogeneous 
groups  (M=28.37)  was  higher  than  the  mean  score  for  homogeneous  groups  (M=25.39). 
High  ability  students  had  a  greater  mean  score  than  low  ability  students  (M=29.73  and 
M=24.01,  respectively).  The  mean  achievement  score  for  students  who  worked  under  the 
learner  control  treatment  (M=27.73)  was  higher  than  the  mean  achievement  score  for  those 
working  under  program  control  (M=25.99). 


966 


3  to 


15 


Insert  Table  1  About  Here 

Three-way  ANOVA  results  demonstrated  that  heterogeneous  grouping  significantly 
improved  achievement  compared  to  homogeneous  grouping  06(1.144)  =10.85,  rcOOI]. 
High  ability  students  outscored  low  ability  students  ©1.144)  =41.78,  *<.001].  Groups 
working  under  learner  control  performed  better  than  those  working  under  program  control 
[£(1,144)  =4.71,  C<.032].  Two-way  interaction  effect  between  Grouping  and  Ability  was 
significant  ffid.144)  =4.85,  p<.029].  The  Scheffe  procedure  indicated  that  homogeneous 
.  low  ability  groups  showed  the  lowest  achievement  compared  to  homogeneous  high  ability 
groups  fec.001)  as  well  as  both  high  fecOOl)  and  low  ability  students  working  together 
in  heterogeneous  groups  0*002).  The  difference  between  the  achievements  high  and  low 
ability  members  of  heterogeneous  groups  also  showed  significance  0*035).  None  of  the 
other  comparisons  were  significant. 

We  also  examined  the  effects  of  independent  variables  on  recall  and  comprehension 
items  separately.  Results  regarding  comprehension  items  suggested  similarity  with  the 
results  for  overall  test  scores.  However,  the  results  on  recall  items  differed  from  the  results 
on  comprehension  items  in  two  important  ways.  First,  the  interaction  between  Grouping 
and  Ability  was  not  significant  Ed,  144)=  1.51,  fi<.221].  Instead,  the  interaction  between 
Grouping  and  Control  showed  significance  E(l,144)=4.30,  B<.040].  Secondly,  three- 
way  interaction  was  significant  [£(1,144)=5.18,  B<.024].  Further  analyses  indicated  that 
the  most  obvious  difference  was  between  the  achievements  of  low  ability  students  working 
under  learner  control  0*001)  in  homogeneous  (M=10.50)  and  heterogeneous  cooperative 
groups  (M=  14.94),  suggesting  the  effectiveness  of  heterogeneity, 
rnnfidence 

The  concept  of  confidence  in  this  study  was  defined  as  the  perceived  likelihood  of 
personal  success  (Keller,  1983).  Upon  completion  of  the  immediate  posttest,  students 
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were  asked  to  predict  their  achievement  score  and  write  it  down  to  the  first  page  of  the  test 
Means  and  standard  deviations  for  confidence  scores  are  presented  in  Table  2.  Students  in 
heterogeneous  groups  had  higher  scores  than  those  in  homogeneous  groups  (M=31.50  and 
M=28.08,  respectively).  High  ability  students  (M-31.20)  reported  greater  confidence  than 
low  ability  students  (M=28.33).  The  mean  confidence  score  for  students  exercising  control 
over  the  lesson  was  slightly  higher  than  the  mean  score  for  those  following  a  predetermined 
instructional  path  (M=30.07  and  M=29.44,  respectively). 


Insert  Table  2  About  Here 


Three-way  ANOVA  results  yielded  significant  main  effects  for  both  Grouping 
E(l,144)=12.58,  fi<.001]  and  Ability  [£(1,144)=  9.05,  p<.003}.  Heterogeneous  groups 
and  high  ability  students  outscored  homogeneous  groups  and  low  ability  students.  Neither 
the  main  effect  for  Control  [£(1, 144)  =0.52,  B<.470]  nor  the  interactions  were  significant. 
Retention 

In  order  to  measure  retention,  an  identical  posttest  was  administered  two  weeks 
after  the  instruction.  Data  from  145  students  were  included  in  the  analysis  since  7  students 
were  absent  during  the  retention  test  The  correlation  between  scores  on  the  immediate  and 
the  retention  posttests  was  .73.  Means  and  standard  deviations  for  retention  scores  are 
contained  in  Table  3.  Students  in  heterogeneous  groups  (M=26.51)  had  higher  retention 
scores  than  those  in  homogeneous  groups  (M=24. 16).  The  mean  score  for  high  ability 
students  was  higher  than  the  mean  score  for  low  ability  students  ((M=28.44  and  M=22. 18, 
respectively).  Students  working  under  learner  control  (M=26.47)  had  higher  retention 
scores  than  those  working  under  program  control  (M=24.01). 


Insert  Table  3  About  Here 
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Thrcc-way  ANOVA  results  demonstrated  that  heterogeneous  groups  retained  more 
information  than  homogeneous  groups  [£(1,137)=5.07,  $><.026].  High  ability  students 
consistently  outperformed  low  ability  students  on  the  retention  posttest  E(l,137)=40.65, 
B<001].  Exercising  learner  control  was  more  effective  for  retention  than  following  a  fixed 
instructional  sequence  l£(l,137)=6.79,  p<.010].  None  of  the  interactions  was  significant. 
Attitudes 

Another  dependent  variable  was  attitudes  toward  the  delivery  system  (CBI),  subject 
matter  (science),  and  group  work  (pairs).  Means  and  standard  deviations  for  total  attitude 
scores  are  included  in  Table  4.  Students  in  heterogeneous  groups  had  a  higher  mean  score 
than  those  in  homogeneous  groups  (M=l  19.95  and  M=l  13.49,  respectively).  The  mean 
score  for  low  ability  students  (M=l  17.56)  was  greater  than  the  mean  score  for  high  ability 
students  (M=l  15.68).  Students  working  under  learner  control  had  a  higher  mean  attitude 
score  than  those  working  under  program  control  (M=120.19  and  M-l  13.26,  respectively). 

Insert  Table  4  About  Here 


Three-way  ANOVA  results  suggested  the  effectiveness  of  heterogeneous  groups 
because  the  main  effect  for  Grouping  was  significant  [E(U44)=5.70,  B<.018].  Although 
there  was  a  difference  in  the  attitudes  of  high  and  low  ability  students,  the  main  effect  for 
Ability  was  not  significant  [Ed,144)=0.57,  ft<.452].  The  main  effect  for  Control  was 
significant  [E(l,144)=6.89,  ft<.010]  in  favor  of  learner  control.  Among  interactions,  only 
the  interaction  between  Grouping  and  Ability  was  significant  QE(1,144)=5.00,  j><.027]. 
The  Scheffe  procedure  indicated  a  statistically  significant  difference  between  the  attitudes  of 
low  ability  students  in  homogeneous  and  heterogeneous  groups  (ji<.015). 

We  also  checked  the  significance  of  results  according  to  categories  of  the  attitude 
questionnaire.  On  the  subscale  for  attitudes  toward  computer-based  instruction,  only  the 
main  effect  for  Grouping  [£(1,144)=5.49,  B<.020]  and  interaction  between  Grouping  and 
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Ability  were  statistically  significant  E(U44)=4.09,  C<.045].  The  Scheffe  procedure 
showed  significance  between  the  attitudes  of  low  ability  students  in  homogeneous  and 
heterogeneous  groups  favoring  heterogeneity  (fi<.025).  No  comparison  was  statistically 
significant  on  the  subscale  for  attitudes  toward  science.  On  the  subcategory  for  attitudes 
toward  group  work,  the  main  affect  for  Grouping  was  significant  E(l,144)=4.27, 
fi<.041].  This  suggested  that  students  in  heterogeneous  groups  developed  more  positive 
attitudes  toward  group  work.  The  main  effect  for  Ability  was  not  statistically  significant 
[Ed  ,144)=0.24,  ft<.627].  Learner  control  promoted  better  attitudes  toward  working  with 
others  in  groups  than  program  control  E(l,144)=14.17,  C<.001].  Two-way  interaction 
between  Grouping  and  Ability  was  the  only  significant  interaction  effect  [E(l,144)=6.78, 
B<.010].  The  Scheffe  procedure  showed  significant  differences  between  the  attitudes  of 
low  ability  students  in  homogeneous  and  heterogeneous  learning  groups  (a<.010)  favoring 
heterogeneity.  No  other  comparison  was  statistically  significant. 

Time  on  Tact- 

Amount  of  time  for  each  cooperative  group  was  recorded  during  the  instruction. 
The  means  and  standard  deviations  for  time  on  task  are  mentioned  in  Table  5.  The  average 
time  for  homogeneous  groups  (M=62.46)  was  longer  than  average  time  for  heterogeneous 
groups  (M-57.89).  High  ability  students  (M=58.88)  spent  less  instructional  time  than  low 
ability  students  (M=61.56).  The  completion  time  was  shorter  for  students  who  exercised 
control  (M=57.32)  than  those  who  followed  a  predetermined  instructional  path  (M=63.00). 


Insert  Table  5  About  Here 


Three-way  ANOVA  results  indicated  that  homogeneous  groups  used  significantly 
longer  time  than  heterogeneous  groups  (E(  1,1 44)=  14.27,  fi<.001].  High  ability  students 
spent  less  time  on  task  than  low  ability  students  [E(l,144)=4.68,  fi<.032].  Program 
control  required  more  time  to  complete  the  lesson  than  learner  control  ffi(l  ,144)=22.60, 
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B<.001].  Two-way  interaction  effect  between  Grouping  and  Ability  was  also  significant 
E(l,144)=4.68,  u<.032].  The  Scheffe  procedure  suggested  efficiency  for  homogeneous 
high  ability  groups  (b<-025)  and  students  in  heterogeneous  teams  compared  to  low  ability 
students  in  homogeneous  groups.  No  other  comparison  suggested  statistically  significant 
differences  regarding  time  on  task. 
Interaction 

Peer  interaction  in  groups  was  coded  according  to  ten  categories  of  the  Interaction 
Scale,  but  only  eight  of  these  categories  were  related  to  verbal  interaction  among  students. 
The  other  two  categories  were  "entering  group  response"  and  "demonstrating  off-task 
behavior".  They  were  excluded  from  comparisons  of  the  treatment  groups  because  these 
behaviors  sometimes  did  not  involve  peer  interaction  or  group  behavior  that  were  directed 
to  completion  of  the  task.  The  means  and  standard  deviations  for  total  verbal  interaction  arc 
reported  in  Table  6.  The  mean  score  for  heterogeneous  groups  (M=l  32.31)  was  higher 
than  the  mean  score  for  homogeneous  groups  (M=100.17).  High  ability  students  averaged 
greater  verbal  interaction  score  than  low  ability  students  (M=134.60  and  M=91.45, 
pectively).  The  mean  score  for  students  working  under  the  learner  control  treatment 
(M=  132.30)  was  higher  than  the  mean  score  for  those  working  under  the  program  control 
condition  (M=93.75). 


a 
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Insert  Table  6  About  Here 


Three-way  ANOVA  yielded  a  significant  main  effect  in  favor  of  heterogeneous 
grouping  Ed.32)=4.44,  o<.043].  The  difference  between  the  interaction  means  of  high 
and  low  ability  students  was  significant  E<1,32)=6.70,  B<.014].  Students  working  under 
learner  control  engaged  in  more  verbal  interaction  than  those  working  under  the  program 
control  treatment  [E0,32)=5.46,  jK-026].  None  of  the  interaction  effects  was  statistically 
significant.  We  also  checked  the  predictability  of  student  achievement  based  on  interaction 
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categories.  The  means,  standard  deviations,  and  correlations  of  interaction  categories  with 
achievement  are  presented  in  Table  7. 


Insert  Table  7  About  Here 


It  appears  that  only  three  categories  of  peer  interaction  can  successfully  predict 
student  achievement  Those  arc  seeking  clarifications  from  the  partner  (r=.3 15,  w<047), 
providing  explanations  after  giving  an  answer  (r=483,  B<.002),  and  demonstrating  off-task 
behavior  during  the  group  work  (r=-445,  c<.004).  It  should  be  noted,  however,  that  the 
correlation  between  off-task  behaviors  and  student  achievement  was  a  negative  one.  When 
total  verbal  interaction  scores  arc  used  to  predict  achievement  on  the  posttest,  the  correlation 
was  positive  and  significant  (r=349,  jj<.027). 

Discussion 

This  study  investigated  the  effects  of  exercising  control  over  the  amount,  review, 
and  sequence  of  instruction  in  mixed  and  uniform  ability  groups  during  a  computer-based 
cooperative  lesson  on  solar  energy.  Dependent  variables  included  achievement,  confidence, 
retention,  attitudes,  time  on  task,  and  interaction. 

Students  in  heterogeneous  cooperative  groups  outperformed  their  counterparts  in 
homogeneous  groups.  The  achievement  differences  were  particularly  noticeable  when  low 
ability  students  in  homogeneous  groups  were  compared  with  others  including  low  ability 
members  of  heterogeneous  teams.  It  appears  that  interaction  among  students  of  discrepant 
abilities  provides  beneficial  opportunities  for  low  ability  students,  and  it  is  not  detrimental 
for  the  achievement  of  high  abitity  students.  This  result  is  consistent  with  the  established 
findings  of  research  on  cooperative  group  composition  (Hooper  &  Hannafin,  1991;  Simsek 
&  Tsai,  1992).  A  possible  interpretation  of  this  result  is  that  low  ability  students  receive 
peer  encouragement  and  personalized  support  from  their  more  able  partners.  They  perceive 
that  their  contributions  are  expected  and  valued  for  success  of  the  group.  Their  partners  are 
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available  to  help  them  when  they  need  a  customized  explanation  for  finding  the  answer  or 
figuring  out  the  solution  process.  When  they  have  an  incorrect  idea,  more  able  students  in 
the  group  con  explain  why  that  answer  is  not  acceptable.  When  the  group  contains  all  low 
ability  students,  on  the  other  hand,  members  usually  share  the  mistakes.  After  getting  the 
feedback  from  the  computer,  they  realize  that  their  response  is  not  an  acceptable  one.  They 
try  to  enter  another  response  without  developing  an  alternative  solution  proposal  because 
low  ability  students  are  usually  unable  to  do  so.  By  continuing  this  simple  tryout  pattern, 
group  members  start  loosing  their  motivation  and  the  perception  that  their  partner  is  able  to 

help  or  offer  different  views. 

Ability  was  an  important  factor  in  achievement  High  ability  students  performed 
significantly  better  than  low  ability  students.  Based  on  a  large  body  of  research,  this  is  not 
a  surprising  result  (Johnson  &  Johnson,  1989).  The  critical  point  is  that  high  ability 
members  of  cooperative  groups  are  usually  more  active  in  group  discussions  contributing 
toward  their  own  success  and  the  success  of  their  partners.  These  students  also  perform  a 
leadership  role  in  heterogeneous  groups  by  providing  personalized  explanations  and  clear 
directions  to  their  low  ability  groupmates.  All  these  activities  involve  deeper  information 
processing  and  metacognitive  skills  including  generative  activities  which  seem  to  contribute 
to  the  achievement  of  more  able  students. 

Students  working  under  learner  control  outperformed  their  counterparts  working 
under  program  control.  This  result  is  somewhat  contrary  to  the  conclusions  derived  from 
studies  on  individualized  instruction  because  there  is  a  tendency  in  the  literature  suggesting 
that  elementary  students  cannot  make  appropriate  decisions  about  their  own  learning  needs 
(Carrier,  1984;  Hannafin,  1984;  Steinberg,  1989).  It  seems  reasonable  to  expect  that  when 
students  do  not  have  metacognitive  skills  necessary  for  effective  decision  making,  they  will 
not  be  able  to  benefit  from  exercising  control  compared  to  following  a  fixed  instructional 
sequence  embedded  in  the  computer  lesson.  However,  the  present  study  suggested  that 
this  explanation  may  not  be  applicable  to  cooperative  learning  since  group  members  usually 
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discuss  the  options  carefully  before  making  a  decision  or  choice.  When  the  lesson  requires 
a  group  consensus  about  each  response,  many  mistakes  can  be  avoided  as  a  consequence 
of  getting  another  opinion.  This  is  not  to  say  that  all  students  will  be  good  decision  makers 
in  cooperative  groups.  A  careful  attention  should  be  given  to  heterogeneity  of  the  group 
and  mastery  contingencies  for  all  students.  Providing  learner  control  option  can  be  harmful 
or  ineffective  for  the  achievement  of  low  ability  students  working  in  homogeneous  groups. 
The  results  of  this  study  concerning  performance  on  recall  items  suggested  that  low  ability 
students  in  homogeneous  groups  did  not  benefit  from  practicing  control.  Each  decision 
was  a  challenge  for  these  students.  However,  it  was  not  the  case  for  low  ability  students  in 
heterogeneous  groups  because  each  decision  served  as  an  opportunity  generating  further 
discussion  and  learning  from  partners.  It  was  suggested  that  such  a  situation  may  motivate 
group  members  to  engage  in  more  discussion  that  results  in  providing  elaborative  feedback 
(Carrier  &  Sales,  1987). 

Results  of  the  study  suggested  that  heterogeneous  grouping  significantly  improved 
student  confidence  about  performance  on  the  posttest  Also,  high  ability  students  were 
more  confident  than  low  ability  students.  However,  exercising  control  over  the  instruction 
did  not  affect  confidence.  These  results  provide  partial  support  for  the  findings  of  several 
other  studies  suggesting  that  student  ability  and  cooperation  in  heterogeneous  small  groups 
increase  confidence,  while  type  of  instructional  control  does  not  have  a  significant  influence 
on  the  outcomes  (Klein  &  Keller,  1990,  Simsek  &  Sales,  1992).  It  may  be  that  low  ability 
students  cooperating  with  equals  in  homogeneous  groups  do  not  have  confidence  cither  in 
themselves  or  their  partners.  Similarly,  high  ability  students  in  homogeneous  groups  may 
lack  the  opportunity  for  comparing  themselves  with  others  because  they  do  not  receive 
ideas  or  solution  proposals  which  are  neither  above  nor  below  their  own  level  of  cognitive 
complexity.  In  heterogeneous  groups,  however,  students  can  compare  their  performance 
with  the  performance  of  their  partners.  High  ability  members  of  the  group  may  feel  good 
about  themselves  because  they  know  more  about  the  task  and  can  provide  others  with  help 
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to  improve  their  performance.  Low  ability  students  in  heterogeneous  group  may  perceive 
that  they  are  contributing  toward  the  mutual  goal  with  the  support  and  encouragement  of 
more  able  partners.  Such  a  perception  may  increase  their  expectations  for  competence  and 
motivate  them  to  invest  more  efforts  directed  toward  group  goals  (Cohen,  1986).  Type  of 
instructional  control  may  not  be  a  strong  determinant  in  this  process  characterized  mostly 
by  group  dynamics  rather  than  whether  the  control  option  is  given  or  not 

Retention  results  suggested  that  heterogeneous  grouping  was  more  effective  than 
homogeneous  grouping  to  help  students  retain  more  information.  A  possible  explanation 
might  be  that  members  of  heterogeneous  groups  benefit  from  a  variety  of  alternatives  and 
explanations  offered  to  solve  a  problem  or  answer  a  question.  When  different  explanations 
are  provided,  students  restructure  their  own  rationale  and  enrich  their  cognitive  reasoning. 
Some  these  strategies  may  help  them  find  clever  ways  of  remembering  or  comprehending 
facts  which  were  the  core  of  the  learning  program  used  in  the  present  study.  This  may  also 
explain  that  high  ability  students  performed  better  on  the  retention  posttest  than  low  ability 
students.  Learner  control  option  along  with  the  requirement  of  reaching  group  consensus 
before  entering  each  response  might  have  provided  additional  opportunities  for  engaging  in 
further  and  elaborated  discussion  promoting  a  higher  retention  rate  than  the  program  control 
treatment 

Students  in  heterogeneous  groups  developed  more  positive  attitudes  than  those  in 
homogeneous  groups.  Low  ability  students  in  homogeneous  groups  reported  the  lowest 
attitude  scores  than  the  other  treatment  groups.  The  difference  in  overall  attitudes  was 
particularly  obvious  when  low  ability  students  in  homogeneous  groups  were  compared 
with  their  low  ability  counterparts  in  heterogeneous  groups.  Moreover,  this  positive  effect 
in  favor  of  heterogeneity  was  not  due  to  an  accompanying  decrement  in  the  attitudes  of  high 
ability  students  cooperating  with  low  ability  students.  These  findings  are  consistent  with 
the  results  of  many  other  research  studies  (Dalton,  Hannafin,  &  Hooper,  1989;  Johnson  & 
Johnson,  1989;  Sharan,  1980;  Simsek  &  Tsai,  1992).  It  may  be  that  low  ability  students 
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in  heterogeneous  groups  feel  more  supported  and  satisfied  than  those  in  other  groups  as  a 
consequence  of  their  interaction  with  more  able  partners.  This  may  produce  a  friendly  and 
motivating  environment  in  which  students  like  the  computer  and  cooperative  group  work 
better,  but  these  feelings  or  perceptions  may  not  be  strong  enough  to  have  direct  influences 
on  the  subject  matter. 

Ability  did  not  affect  attitudes.  Perhaps,  affective  outcomes  as  operationalized  in 
this  study  cannot  easily  be  influenced  by  aptitude  variables.  Another  possibility  is  that 
attitudes  toward  the  delivery  system,  subject  matter,  and  group  work  arc  mostly  situational 
and  may  not  be  positively  correlated  with  student  ability.  Also,  the  interaction  between 
Grouping  and  Ability  in  favor  of  low  ability  members  of  heterogeneous  groups  may  have 
reduced  the  possibility  for  finding  a  significant  main  effect  for  ability. 

Students  exercising  control  over  the  lesson  developed  more  positive  attitudes  than 
those  following  a  predetermined  instructional  path.  This  result  is  supportive  of  the  findings 
reported  by  Simsek  and  Sales  (1992).  Students  exercising  control  over  the  lesson  may  feel 
that  they  have  certain  amount  of  freedom,  choice,  and  flexibility  in  the  learning  process. 
They  may  also  think  that  they  are  the  ones  making  important  decisions  and  their  decisions 
are  respected.  Given  an  option  to  decide  whether  to  receive  more  instruction,  skip  certain 
parts  of  the  lesson,  or  terminate  the  instruction  may  create  intrinsic  motivation  and  promote 
more  positive  attitudes  toward  components  of  that  particular  learning  environment  (Carrier 
&  Sales,  1987;  Kinrie,  1990;  Steinberg,  1989). 

Results  regarding  time  on  task  demonstrated  that  heterogeneous  groups  and  learner 
control  subjects  used  significantly  less  instructional  time  than  homogeneous  groups  and 
program  control  subjects.  The  main  effect  for  Ability  was  also  significant  Analysis  of  the 
interaction  between  Grouping  and  Ability  showed  that  low  ability  students  in  homogeneous 
groups  consistently  needed  more  time  to  complete  the  lesson  than  the  other  groups.  These 
results  support  the  efficiency  of  cooperative  learning  for  less  able  students  in  heterogeneous 
groups  (Hooper  &  Hannafin,  1991;  Simsek  &  Hooper,  1992;  Simsek  &  Tsai,  1992;  Webb 
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&  Cullian,  1983).  The  reason  might  be  that  students  in  heterogeneous  groups  develop 
higher  understanding  of  the  task  and  employ  more  effective  strategies  in  accomplishing  the 
mutual  goals.  As  a  consequence  of  interacting  with  more  able  partners,  low  ability  students 
in  heterogeneous  groups  may  acquire  better  problem  solving  skills.  Using  more  efficient 
strategies  may  save  important  amount  of  time  for  these  students.  Low  ability  students  in 
homogeneous  groups,  however,  may  not  benefit  from  diversity  in  generating  solutions  that 
help  them  avoid  typical  mistakes  and  time  consumption.  The  efficiency  of  learner  control 
may  be  attributed  to  exercising  control  over  the  amount  of  instruction  as  well  as  using  the 

review  option  effectively. 

The  interaction  results  are  in  agreement  with  the  findings  of  other  relevant  studies 
(Hooper,  1992;  King,  1989;  Webb,  1983).  When  peer  interaction  in  groups  was  analyzed 
based  on  certain  categories  of  interaction  behaviors,  providing  elaborations  after  giving  an 
answer  «-s  found  to  be  a  successful  predictor  of  student  achievement.  Another  category 
which  seem  to  be  a  good  predictor  of  achievement  was  seeking  clarification  after  receiving 
an  answer  or  explanation  from  the  partner.  The  correlations  between  these  two  categories 
and  achievement  were  both  positive  and  statistically  significant.  The  final  category  which 
had  the  potential  to  predict  achievement  successfully  was  demonstrating  off-task  behavior 
during  the  group  work.  However,  the  correlation  between  this  category  and  achievement 
was  a  negative  one.  That  is,  any  increase  in  off-task  behavior  accompanied  a  decrease  in 
achievement  scores  on  the  posttest 

When  the  categories  related  to  verbal  interaction  among  students  wrw  combined 
and  the  significance  of  differences  among  treatment  groups  were  compared,  all  independent 
variables  showed  significant  main  effects,  suggesting  the  effectiveness  of  heterogeneous 
grouping,  higher  ability,  and  learner  control.  It  is  reasonable  to  expect  that  when  students 
are  to  make  selections  or  decisions,  thoy  engage  in  -note  interactions  compared  to  those 
following  a  fixed  sequence  of  instruction.  It  is  also  reasonable  to  assume  that  members  of 
heterogeneous  groups  demonstrate  greater  exchange  of  ideas  as  a  consequence  of  trying  to 
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convince  their  partners  with  diverse  abilities.  Students  in  homogeneous  groups,  however, 
may  have  more  agreements  than  disagreements  as  a  result  of  their  similarities.  This  makes 
discussions  or  explanations  unnecessary,  but  students  benefit  from  explaining  even  in  the 
case  agreement  with  their  partners  because  explanations  may  clarify  misunderstandings  as 
well  as  potential  conflicts. 

The  results  of  this  study  have  certain  implications  for  both  instructional  designers 
and  educational  researchers.  It  appears  that  heterogeneous  cooperative  grouping  is  an 
effective,  appealing,  efficient  alternative  to  individualized  instruction.  This  kind  of  learning 
particularly  benefits  low  ability  students,  and  it  is  not  detrimental  for  high  ability  students. 
Most  of  the  problems  associated  with  learner  control  can  be  overcomed  when  members  of 
heterogeneous  groups  make  the  decisions  together.  Diversity  among  these  students  can  be 
a  motivating  factor  when  students  seek  and  provide  explanations.  It  seems  likely  that  high 
ability  students  will  dominate  group  interaction,  but  a  knowledgeable  instructional  designer 
can  incorporate  certain  elements  providing  equal  opportunities  for  participation  of  all  group 
members.  Such  important  features  of  computer-based  instructional  lessons  may  include 
requiring  a  consensus  for  each  response,  encouraging  students  to  explain  their  rationales 
behind  decisions,  assigning  critical  roles  to  members,  checking  randomly  if  everyone  has 
agreed  on  the  choices  made,  giving  multiple  opportunities  for  mastery  before  proceeding  to 
the  next  practice  item  or  lesson  segment,  asking  students  to  take  turns  using  input  devices, 
and  providing  specific  instructions  for  effective  group  processing. 

Further  research  is  needed  to  clarify  the  relationship  between  group  composition 
with  regard  to  students*  learning  styles  and  academic  performance.  Integrating  generative 
activities  into  cooperative  decision  making  should  be  further  investigated  The  effects  of 
various  feedback  strategies  during  cooperative  group  work  needs  to  be  examined,  The 
relationship  between  exercising  learner  control  and  man-  Jlating  group  size  deserves  more 
research  attention.  Qualitative  studies  may  provide  better  insights  than  quantitative  studies 
in  analyzing  peer  interaction  in  cooperative  groups. 
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Table  1 .  Means  and  Standard  Deviations  for  Achievement  Scores 


32 


Type  of  control 

AhUity                                  Learner          Program  Total 

Homogenftniifi 

High 

ctL                                 30.78             27.85  29.24 

N.                                    A'52               5-63  5.28 

•                                   18                20  38 

Low 

Sn-                                 2i  i5             22.30  21.73 

SP:                                  5-82              6.11  5.92 

N:                                   20                20  40 

Total 

SD-                                       25n1\               2|°8  2539 

N                                     ZU               644  6.74 

N-                                   38                40  78 

Ht5tempanftnii?t 

High 

J*  31.28  29.26 


«n.  -----  30.24 

£D'  ,i94  5.45  5.24 

N-  18  19 


Low 


5*  28.44  24.63 

£D:  3.71  6.25 

•  18  19 

Total 


N: 

High 

M: 
SD: 
N: 


Low 


N:  38  39 

Total 


M: 

SD: 

N: 


37 

26.49 
5.46 
37 

28.37 


SD-  29A6a  2695 

m.                                     4-54  6.24  5.64 

36  38  74 

Camlaned 

31.03  28.54  29  73 

4.68  5.52  5.25 

36  39  75 

24.61  23.44  24.01 


M: 

m.  6.21  6.15 

77 


27.73  25.99  26.84 

6.31  6.37  6.38 

74  78  152 
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Tabic  2.  Means  and  Standard  Deviations  for  Confidence  Scores 
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Ability 


SD: 
N: 


SD: 
N: 


N: 


ERIC 


Type  nf  control 
Lcamer  Program  Total 


Hnrngftrienus 

High  30.72  28.65  29.63 

SD  404  533  4  82 

N:  18  20  38 

Lo*  26.45  26.75  26.60 

TO-  8.00  7.25  7-54 

SD-  20  20  40 

Total  2g47  27.70  28.08 

TO-  671              6.36  650 

g?'  38  40  78 

mghM.  33.06  32.58  32.81 

JK.  3.92              4.56  4.21 

g?"  18  19  37 

Low  30.44  29.95  30.19 

SD-  "61              378  ,5V88 

ST  18                 19  37 

Total  31?5  3L26  31.50 

5*.  6*11              4.34  5.25 


36  38  74 

Combined 


High  31g9  30.56  31.20 

4.10  5.30  4.77 

36  39  75 


w  w.                                 28.34  28.31  28.33 

en.                                  7.97  5.97  6.99 

38  39  77 

TOtalM:                                  30.07  29.44  29.74 

SD-                                  659  5o72  io15 

kiV"                                 74  78  152 
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Table  3.  Means  and  Standard  Deviations  for  Retention  Scores 


Tvpeofcnntml 

Ability  Learner  Program 


Low 

M: 
SD: 

N:  37"  37 

Total 

J*  26.47  24.01 


Total 


Homoyenenng 

High 

J*  29.94  27.22  28.08 

£D:  6.84  5.56  6.20 

N:  18  18  36 

Low 

5*  20.75  20.50  20.63 

SP-  546  6.45  5.90 

N:  20  20  40 

Total 

24.63  23.68  24.16 

£D-  7.35  6.87  7.08 

N;  38  38  76 

Heterogeneity 

High 

SrV  3i-Z5  2658  28-94 

ST  6.31  6.88 

N-  i6  ir  35 

Low 

J*  25.59  22.41  24.00 

XT'  3,54  6.05  5.14 

N:  17  17  34 

Total 

SD-  2%A\  2461  26.51 

N  *05  6.46  6.53 

N>  33  36  69 

Combined 

High 

£1  30.27  26.72  28.44 

v?'  6-79  5.87  6.54 

N>  34  36  70 


SD 
N 


22.97  21.38  22.18 

5.22  6.26  5.78 

74 


25  28 

7.01  6.60  6.90 

71  73  145 
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Tabic  4.  Means  and  Standard  Deviations  for  Attitude  Scores 


Typ»"f  control 

Ability  learner  Program  Total 


Hnpinyeneous 

High 

M: 

SD: 

N: 

119.11 

1  A  AC 

10.95 
18 

112.15 
17  54 

20 

115.45 
15.02 
38 

Low 

M: 

SD: 

N: 

112.65 
19  58 
20 

110.60 
23.44 
20 

111.63 
21.34 
40 

Total 

M: 

SD: 

N: 

115.71 
16  20 
38 

111.38 
20.45 
40 

ffrtrmfreneous 

113.49 
18.51 
78 

High 

M 

SD: 
N: 

120.44 
10.48 
18 

111.63 
15.81 
19 

115.92 
14.03 
37 

Low 

M: 

SD: 

N: 

129.39 
11.53 
18 

118.84 
17.69 
19 

123.97 
15.74 
37 

Total 

M 

SD: 
N: 

124.92 
11.77 
36 

115.24 
16.95 
38 

Combined 

119.95 
15.35 
74 

High 

M: 

SD: 

N: 

119.78 
10.58 
36 

111.90 
16.50 
39 

115.68 
14.44 
75 

Low 

M: 

SD: 

N: 

120.58 
18.16 
38 

114.62 
20.99 
39 

117.56 
19.74 
77 

Total 

M: 

SD: 

N- 

120.19 
14.86 
74 

113.26 
18.81 
78 

116.63 
17.30 
152 

•87  y.r; 
u  u  u 
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Tabic  5.  Means  and  Standard  Deviations  for  Instructional  Time 


Type  of  control 

Ability  Learner  Program  Total 

Homogeneous 

High 

M:  58.1i  61.40  59.84 

J*  9.68  5.86  7.97 

N:  18  20  38 

Low 

M:  60.60  69.30  64.95 

Sp  8.83  5.83  8.60 

N:  20  20  40 

Total 

M  59.42  65.35  62.46 

9.20  7.02  8.63 

N:  38  40  78 

Heterogeneous 

High 

M  55.11  60.53  57.89 

5*  6.72  7.37  7.49 

N:  18  19  37 

Low 

M  55.11  60.53  57.89 

5*  6.72  6.72  7.49 

N:  18  19  37 

Total 

M  55.11  60.53  57.89 

5*  6.62  7.27  7.44 

N:  36  38  74 

Combined 

High 

J*  56.61  60.97  58.88 

JTD:  8-35  6.57  7.74 

N:  36  39  75 

Low 

M  58.00  65.03  61.56 

5*  828  7.90  8.78 

N:  38  39  77 

Total 

J*  57.32  63.00  60.24 

8TD:  8.29  7.50  8.37 

N:  74  78  152 
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Tabic  6.  Means  and  Standard  Deviations  for  Verbal  Interaction 


Ability 


High 


Low 


Total 


M: 
SD: 
N: 

M: 

SD: 

N: 

M: 

SD: 

N: 


High 


Low 


Total 


M: 
SD: 
N: 

M: 

SD: 

N: 

M: 

SD: 

N: 


High 


Low 


Total 


M 

SD: 
N: 

M: 

SD: 

N: 

M: 

SD: 

N: 


Typ*  of  control 
Learner  Program 


163.00 
57.98 
6 

87.50 
63.65 
6 

125.25 
70.18 
12 


160.25 
51.91 
4 

125.50 
46.60 
4 

142.88 
49.30 
8 


161.90 
52  61 
10 

102.70 
57.97 
10 

132.30 
61.85 
20 


Homogeneous 

95.17 
37.11 
6 

55.00 
19.95 
6 

75.08 
35.31 
12 

Tfr»>mgcneous 

125.50 
40.01 
4 

118.00 
45.28 
4 

121.75 
39.76 
8 


Total 


129.08 
58.39 
12 

71.25 
48.07 
12 

100.17 
60.07 
24 


142.88 
46.75 
8 

121.75 
42.72 
8 

132.31 
44.62 
16 


107.30 

134.60 

39.29 

53.17 

10 

20 

80.20 

91.45 

44.30 

51.52 

10 

20 

93.75 

113.03 

43.06 

56.11 

20 

40 
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Table  7.  Means,  Standard  Deviations,  Correlations  with  Achievement,  and  Probabilities  of 
Interaction  Categories 


\A 

m 

or* 
bu 

r 

P 

Reading  information 

ZO.  JO 

.275 

.085 

A  skin  p  Questions 

in 

O.Jo 

.178 

.273 

Giving  answers 

1  <  IK 
ID.  JO 

.220 

.173 

Seeking  clarification 

Z.0O 

0  Ofl 

.315 

.047 

Providing  explanations 

15.33 

15.14 

.483 

.002 

Making  comments 

5.60 

4.19 

-.099 

.542 

Energizing  the  group 

13.93 

11.48 

.224 

.164 

Suggesting  directions 

7.58 

5.38 

-.120 

.462 

Entering  group  response 

36.80 

23.89 

.227 

.159 

Showing  off-task  behavior 

9.73 

13.50 

-.445 

.004 

Verbal  interaction 

113.03 

56.11 

.349 

.027 

Total  group  discourse 

159.55 

70.00 

.271 

.090 
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™*  Cognitive  modeling  it.  a  method  by  which  data  on  human 

performance  is  quantified  and  interpreted.    In  the  field  of 
human-computer  interaction,  cognitive  models  provide  theoretical 
constructs  that  attempt  to  identify  the  mental  processing 
required  to  perform  a  given  class  of  computer  tasks,  and  which 

S^JSrSSLrgS* i02S  '°;.Y«ri«tioS.  in  performanceWac?5ss 
users  and  across  tasks.    Many  different  types  of  models  exist  in 
the  current  human-computer  literature  (c.f.  Booth,  1989;  Slson 
and  Olson,  1990),  and  have  been  successfully  applied  to  a  ranSe 
of  computer-based  tasks,  such  as  word-process iSg  or  automatic"- 
teller  usage,     it  is  often  argued  that  this  typl  of  modeling 

p^Jo^anc^w^f.r0'113^6  Way  t0  assess  andpreSiSt  SiiSn 
?n«fSS  ?  ?    tlh  lhB  comPuter'  and  allows  the  most  penetrating 
in^e?actto£°  Ct°rS  that  g°Vern  •"•<*iv«  huma£-compute£9 


charac£erf  ^S^Y?  ™°*eling  has  been  a  useful  technique  for 
cnaracterizing  adult  computer  use,  the  suitability  of  this  method 
SE  22£"£Xl2  ^%Peff°rmance  of  young  chil^enls^imUe^0'1 

SLec?ed°SteS  UT<  that  tL  --.fp^fo^nce  1??™™* 

mZ~Z      *  %*rd  solvinq  *  clearly  defined  goal  (Booth,  1989; 
Olson  and  Olson    1990).    Casual  use,  where  there  is  no  clear 
goal,  or  use  behavior  that  is  filled  with  errors  or  digressions 
i!h"5?  ^-^Ptured  by  cognitive  models.     Un fortunate ?£?  t^h 
to  ^t?  13  -^iCal  °f  Presch°ol  children,  and  thus  would  Appear 

"       °°gniilve  modeling  unsuitable  for  understanding  the 
performance  of  such  children  when  using  a  computer. 

™n  J??iP^eSent  Papfr  takes  the  Position  that  cognitive  modeling 
can  still  be  a  useful  strategy  for  organizing  data  on  children 
performance  with  computers  if  the  domain  being  studied  is 
n^Eriate-y  d?ffned«    The  current  model  is  concerned  with  the 
narrow  domain  of  interface  devices  for  cursor  control    and  their 
Soff/;  young  children  as  computer  users.    ?SfpurposS  of  t£e 

sevJrai  s^ud?oSVo?e  ^i"^*1  exPla"ation  for  theresults  of 
several  studies  of  children's  use  of  a  variety  of  different  irm,n- 
devices,  conducted  in  the  past  five  years  at  Chi Idre"s  P 
Television  Workshop.    This  explanation  should  b.  warded  as 

^ri^ri^r" studiM  wni     to  theses 
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The  model 


:rlc 


The  model  begins  with  the  view  that  cognitive  developme nt  is 

ass aAr^1- «  s^sts^s  s  n 

a  given  task-snd/or  (b)  they  may  Know  all  the  "jcessary  concepts 
let  lack  the  mental  storage  space  to  eotlvate =*h^h2  J ocSS  o*  the 
Wpreesente^k,Taok  ££  SlZWXllt™  2£i£E?? 
SSS  "S.^  S=K  r?.ik^irfngaoo  !?ive 
e?fort  -  oS?  they  may  also  l.ok  the  working  memory  to  actively 
utilize  all  the  knowledge  they  do  have.  _,,<  ,*.-,>„'» 

The  domain  of  behavior  being  deecribed  is  children  s 
comoetence  of  use,  both  initially  and  after  practice,  of 
different  'So inting"  devices,  all  of  which  have  the  common 
Munitions  o?° U?  dxrectin,  a ' cursor  to  ■^^"tro*."?"' 
anH  o\  activatinq  those  icons  via  a  confirmatory  Keystroke  or 
SCSton  D?ess!    This  behavior  is  considered  to  have  two  additive 
psychological  components:    The  cognitive  demands  o  cursor 

control  as  a  behavior,  and  the  P"graaJic/e™a"^'H°Ln^U;se' 
the  hardware  device  itself  that  must  be  followed  during  use. 
S?h  or  these  components  require  the  activation  and  use  of 
Information  that  must  be  either  learned  or  drawn  from  long-term 
memo™?  anS  brth  components  require  working  memory  during  their 
activation  and  use. 

cognitive  demands  gj  nursor  control  _.tural 
This  model  begins  from  the  assumption  that  the  most  J^ral 
way  for  a  Joung  child  to  make  choices  via  a  graphic  "terface  is 
through  physical  actions  that  are  already  natural  and  familiar  to 
the  child,  in  this  case:  pointing.     Pointing  to  pictures  in 
books,  or  to  desired  objects  in  the  store,  school ,  or  home ,  is  a 
behavior  established  in  infancy.     It  is  assumed  here  that 
pointing  requires  no  cognitive  effort  on  the  child's  part.  It  is 
11  wSSatiS  motor  behavior  whose  execution  is  completely 
routinized    with  no  conscious  effort  required  for  its 
performance.    S  other  words,  no  working  memory  is  required  when 
4-hp  child  onlv  needs  to  point  to  a  choice. 

the  CSos?  compu?er  input  devices  take  advantage  of  Pointing  as  an 
easy,  swift  method  for  making  choices.     Evaluations  of  the  design 
and  properties  of  computer  pointing  *«vices  recede Ji^L^f 
cursor  control  devices  usually  conserve  various  co»^nat£?ns.  °f 
the  three  fundamental  properties  of  human  »w~n*J^*  pl»*  ! . 
role  in  pointing:  direction,  distance,  and  SE^SA  (Buxton,  1986, 
Mackin?ay?  2ar2?  and  Robertson ,  1990).    The  mouse,  for  example, 
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conserves  all  three  properties  of  the  user's  physical  movement  in 
the  cursor.    The  mouse's  motion  on  the  tabletop  is  precisely 
mimicked  on  the  screen  by  that  of  the  cursor,  and  positioning  the 
mouse  is  equivalent  to  pointing  a  finger  at  a  choice.  The 
joystick,  in  contrast,  only  utilizes  the  direction  of  the  user's 
movement.     Speed  is  fixed  by  the  machine,  and  distance  is 
determined  by  how  long  the  shaft  is  pressed  -  not  by  any  actual 
distance  of  the  user's  movement. 

For  the  present  paper,  it  is  argued  that  the  less  a  pointing 
device  conserves  the  three  properties  of  movement,  the  more 
demanding  it  will  be  for  young  children  to  use.     This  increase  in 
difficulty  as  devices  become  less  like  pointing  is  hypothesized 
to  be  due  to  the  fact  that  children  must  exert  more  mental 
K2C!.^Sing  elf0rt  t(?  anticiP*te,  and  evaluate,  the  movement  of 
SS!*T5fWr  *SelV t?elr  °Wn  movements  are  not  conserved  by  it,  and 
that  this  effort  increases  with  each  property  of  their  own 
movement  that  is  not  conserved  by  the  input  device.     In  a  sense, 
the  less  the  cursor  responds  directly  to  their  own  actions,  the 
more  it  moves  "independently"  of  them,  and  they  therefore  must 
?h„wC2  mfntal  resources  to  monitoring  its  movement  than 

they  would  if  it  were  completely  mapped  onto  their  own  movements. 
The  model  assumes  that  the  mental  resources  (attention  to  cursor 
position  and  motion,  planning  of  cursor  movement,  etc.)  required 
aren«?Snly?^n?°VlCH  does"°t  conserve  the  child's  own  movement 
2l2n?;?S;£i?      i\2nd  relativelV  stable  -  they  do  not  diminish 
significantly  with  experience. 

Rules  of  device  dsp 

While  the  cursor's  movement  on  the  screen  must  be  attended 
to»  tnere  is  another  demand  the  child  must  attend  to:     The  rules 
of  device  use.     Every  input  device  must  be  used  in  a  specific 

Th!!?eLoJ  i    1S  *°  ^J*3?*  successfully.     The  particular  actions 
that  must  be  performed  with  the  device  to  move  the  cursor 

ioM«i«g/2  m°?Se'  r°U!n3  a  trackball,  etc.),  other  specific 
SSJi™  M  !epJ?g  3  dew1(?e  ln  tne  correct  orientation,  pressing  a 

an    a  ?:1?!!  a  choice)  ,  and  other  cognitive  elements  (such 
as  translating  the  directions  of  movement  of  the  device  to  the 
cursor  on  the  screen)  vary  with  the  device  used. 

This  model  presents  the  hypothesis  that  the  more  such  rules 
a  device  requires  for  use,  the  more  difficult  it  will  be  for 
SiiJiTliSiZS  rSi-y         efficiently,  because  they  consume  the 
Si  S.Lii!    *    WorKin<?  memory  capacity  in  a  manner  similar  to 
the  demands  of  cursor  control  itself.     However,  unlike  the 
cognitive  demands  of  cursor  control  described  above,  this  model 

ST^iJSt*  ?r  -cvul;; differ  frora  cursor  contro1 

ways.     First,  it  is  hypothesized  that  these  rules  require  less 
resources  than  cursor  control.     Second,  unlike  curso?  control 
device  rules  are  easily  assimilated  with  experience?  and  they 
quickly  become  "chunked,"  or  automated  (Case,   1985?    ceasing  to 
require  active  cognitive  processing  when  they  are  b4iSg  invoked 
in  other  words,  repeated  practice  with  the  rules  of  device  use 
quickly  causes  them  to  become  habitual  and  unconscious^  so  Sat 
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the  same  set  of  rules  comes  to  require  less  cognitive  effort  over 
time  than  they  required  initially. 

peifliifip  betw^n  demands.. of  qurior  control find  device  j™*;* 

The  model  is  based  on  the  hypothesis  that  the  demands  on 
workina  memorv  bv  both  cursor  control  and  dsvice  rules  are 
additive     That  is,  th«  combination  of  the  mental  effort  required 
JfcSor  control  and  that  required  for  remembering  all  devices 
rSles  S£s  to  the  total  demand  that  a  particular  device  makes  of 
Si  Shild  user      It  is  hypothesized  that  children's  competence 
£i?h  £  given  input  device  is  impaired  when  the  combined  load  on 
working  memory  of  cursor  control  and  device  rules  exceeds  the 
phiiHic  current  working  memory  capacity. 

An  additional  hypothesis  is  that  when  the  extra  demands  of 
fho  ai?„al  software  task  itself  (solving  math  problems,  finding 
2^IS?lit5r.!  Sc!)?  combined  with  the  demands  of  the  device, 
exceed  the  capacity  of  the  child's  working  memory,  the  child's 
SiUty  to  control  the  cursor  will  be  impaired  before  the  child  s 
ability  to  solve  the  problems  presented  by  the  sof tware . 
is    the  child  will  still  be  able  to  solve  the  problem  presented 
S'tne  compu?e^  out  will  be  unable  to  respond  effectively  using 
the  input  device. 

Performance  explained  by  the  model 

in  a  series  of  studies  comparing  children's  use  of  various 
input  Sevices  tlevelle  and  Strommen,  1990;  Revelle,  Strommen,  and 
Overman!  SSo    Strommen,  1993),  and  in  research  on  specific 
devices  as  well  (Strommen,  1992;  Strommen,  Razavi,  and  M ed off, 
?992;  Strommen  and  Revelle,  1990),  the  results  indicated  a 
consistent  ranking  of  devices  in  terms  of  the  ease  of  witn  wnicn 
ch    drencanuse  the  devices  to  place  a  cursor  on  an  icon  on  the 
television  screen  (ease  is  defined  both  in  terms  of  lengt h  of 
time  required  to  place  the  cursor,  and  in  terms  of  total  J^urate 
cursor  Placements)  .     This  rank  ordering  is  shown  in  both  Tables  1 
and  2,  aloSg  within  initial  analysis  of  the  cognitive  demands  of 
K«*-h  rursor  control  and  use  rules  for  each  input  device.  This 
o?dSring  is  consistent  wilh  the  hypothesis  that  the  more  removed 
a  device  gets  from  the  actual  physical  act  of  P^ing  by  the 
child,  the  more  cognitive  resources  are  required  by  the  child  to 

USS  ^niddiUon^the  inclusion  of  device  rules  explains  a  unique 
finding  in  the  three  major  studies:  children's  performance  with 
the  trackball  is  consistently  superior  to  that  the  mouse  m  the 

e    S      vSOf  use.     Note  that  the  trackball  and  the  mouse 
botrconsegve^all  three  of  the  dimensions  of  the  gild's  own 
actions,  suggesting  that  they  make  the  same  demands  of  the  child 
In  te?ms  of  cursor  control.    However,  the  mouse  has  many  more 
p?agma?ic  rules  associated  with  its  use.    These  rules  hinder 
children  initially,  but  after  several  days  of  P"^ ice  their 
competence  with  the  mouse  quickly  rises  to  the  high  level  of 
performance  seen  right  from  the  start  with  the  trackball. 
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Current  issues  confronting  the  model 

Level  of  detail  of  the  analyses 

The  current  model  rests  on  an  assessment  of  the  cognitive 
demands  of  cursor  control  and  device  use  rules.     A  persistent 
problem  with  cognitive  modeling  techniques  is  determining  the 
appropriate  level  of  detail  that  the  model  needs  to  specify  in 
order  to  be  accurate:  "...the  most  common  predictor  of  cognitive 
complexity  is  the  number  of  rules  that  are  required  to  describe  a 
task.    Unfortunately,  the  number  of  rules  that  are  generated  by 
any  modelling  technique  may  be  dependent  not  only  upon  the  true 
complexity  of  the  task,  but  also  upon  the  grain  of  the  analysis 
employed  (Booth,  1989,  p. 92)."    it  is  not  clear  that  the  analysis 
of  the  devica  rules,  or  the  analysis  of  cursor  control  in  terms 
of  the  three  dimensions  of  pointing  behavior,   is  sufficiently 
detailed  or  is  too  detailed.    The  validity  of  the  pragmatic 
device  rules,   in  particular,  needs  to  be  more  carefully 
investigated. 

A  second  issue  is  determining  the  true  cognitive  load  that 
the  various  forms  of  cursor  control  actually  impose,  and  the  true 
cognitive  load  of  the  device  rules,  as  well.     The  model  current 
assumes  that  the  cognitive  demands  of  cursor  control  increase  in 
direct  ratio  to  the  loss  of  movement  dimensions  conserved.  For 
example,  if  device  rules  were  all  equal,  it  is  assumed  that  the 
mouse  and  trackball  would  make  equal  memory  demands,  and  that 
both  would  make  equally  less  demands  than  the  joystick,  which 
conserves  only  one  aspect  of  movement  instead  of  all  three. 
Similarly,  the  model  currently  assumes  not  only  that  the  device 
rules  are  all  equal  in  their  memory  demands,  but  that  they 
require  less  cognitive  resources  than  cursor  control  as  a  mental 
act.    How  much  less  demand  they  make  is  not  known.     All  these 
assumptions  need  to  be  experimentally  verified. 

Finally,  research  that  links  working  memory  capacity  to 
children's  competence  when  using  different  input  devices  needs  to 
be  conducted.     The  current  model  relies  on  a  specific  theory  of 
cognitive  development  that  quantifies  working  memory  in 
measurable  ways,     if  the  present  model  is  correct,  it  should  be 
possible  to  associate  children's  measured  working  memory  with 
their  performance  using  different  input  devices. 

In  conclusion  the  current  model  shows  theoretical  promise  as 
a  framework  for  analyzing  children's  performances  with  different 
input  devices,  and  explaining  and  predicting  differences  in 
competence  not  only  across  devices  but  across  children  (with 
differing  working  memory  capacities)  as  well.     it  has  the 
potential  to  provide  the  first  specific  conceptual  link  between 
developmental  psychological  theories  and  children's  ability  to 
use  interactive  technologies,  by  incorporating  competence  with 
input  devices  into  an  existing  theoretical  framework.     The  model 
is  inadequately  specified  at  present,  and  specific  empirical 
tests  of  its  constituent  features  is  needed. 
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Table  1  Cursor  control  demands  of  standard  input  devices, 
listed  by  overall  ease  of  use  by  preschoolers. 


Device  Cursor  control  demands 


Touchscreen 
Light  pen 
Trackball 

Mouse 

Joystick 
Arrow  keys 


None:  identical  to  physical  act  of  pointing 

None:  same  as  pointing,  but  with  an  object 
as  the  pointer 

Minimal:  Speed,  duration,  and  direction  of  ball 
rotation  (child's  movement)  translate  directly 
into  cursor  movement. 

Minimal:  Spaed,  duration,  and  direction  of  mouse 
movement  on  surface  (child' b  movement)  translate 
directly  into  cursor  movement 

Moderate/Heavy:     Direction  is  only  aspect  of 
physical  movement  conserved  by  device 

Heavy:     No  aspect  of  movement  is  conserved 
by  device(s) . 

Directions  are  separated  into  unique 
keys  requiring  extra  step  of  planning 
of  movement  by  user. 
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Table  2.  Rules  of  use  of  standard  input  devices,  listed  by 
overall  ease  of  use  by  preschoolers. 


Device 


Device-specific  use  rules 


Touchscreen 
Light  pen 

Trackball 


Mouse 


Joystick 


Arrow  keys 


1, 
2, 

1, 
2, 


1, 
2, 
3, 


2, 
3, 
4, 
5, 
6. 


7. 


1. 
3, 


1. 
2, 


Touch  desired  icon  with  finger 
Press  button  or  key  to  confirm 

Touch  tip  of  pen  to  desired  icon 

Press  button  or  press  pen  on  icon  again 

to  confirm 

Roll  ball  to  select  desired  icon  with  cursor 

Press  button  to  confirm 

Moving  cursor  "up"  and  "down"  on  screen 

means  rolling  ball  away  from  and 

toward  you. 

Slide  mouse  in  desired  direction  of  cursor 

movement,  stopping  on  desired  icon 

Press  button  on  mouse  to  confirm 

Do  not  move  mouse  while  pressing  button 

Keep  mouse  flat  on  surface 

Keep  mouse  in  correct  orientation 

Moving  cursor  "up"  and  "down"  on  screen 

means  sliding  mouse  away  from  and 

toward  you 

At  edge  of  table,  lift  mouse  and  replace  in 
center  to  continue  cursor  movement 

Press  control  shaft  to  move  cursor  to  icon 
release  shaft  to  stop 
Press  button  to  confirm 

Moving  cursor  "up"  and  "down"  means  pressing 
shaft  away  from  and  toward  you 

Press  appropriate  keys  to  move  cursor  to 
icon 

Press  ENTER  to  confirm 


:rjc 


998 


Is  J  ( 


8 

References 


p    fi9B<n     &n  introducer  feg  hur^n-^omputer -j DfcfiEflSfciaii' 

Hillsdale? * N J : ^vrence  Erlbaum  Associates';  Publishes: 

Buxton,  W.   (1986).  There's  more  to  interaction  than^jeets  the 
pve.  some  issues  in  manual  input,     in  D.  a.  woruwu  « 
SJaper  "Ids.),  User  centered  sys^geslgn  <PP-  319-337). 
Lawrence  Erlbaum  Associates,  Hillsdale,  NJ. 

Case,  R.    (1985).     Tptrl]— 1  ^V1mt!  Rirt"n  t0  ^ulttlgod. 
' Orlando,  FL:  Academic  Press. 
,  •  ,        -r      n*r*    9    K      &  Robertson,  G.  G.   (1990)  A  semantic 
"■^SKj.u'oFS;  sp^ofof  input  devices.  Hyjnanz 

Comp"*-°*-  interaction,   5,   145  -  190. 

cv^piH-ar  Infraction,   5.,   221  -  265. 

t    t  T-ia7    h     ri989).  Mental  capacity  testing  as  a 

Publications. 

*"„rn»1  of  CorT"H™  <"  Childhood  Education,  Zd),  33-41. 

Revelle    G.  L. ,  Strommen,  E. ,  &  Offerman,  S.   (1990).  CursojL^M 
ReVS  ^fer-r~g  In  nMldrrn's  compter  control, 

Unpublished  manuscript. 

controller.     »rr1iffl  F-rcmnomics,  22,  409-413. 

r^nnlngy  F»g»»rch  and  Development,  2£(4),   65  80. 

ofrnfflBon    E    F     M992).  Formative  studies  in  the  development  of  a 
Str°mnew' colter  poising  device  for  young  children.  E&jcaj^oj^ 
Technology,  12.(4),  43-51. 

Presented  a?  t£e  Annual  Meeting  of  the  American  Educational 
Research  Association,  Atlanta,  GA. 


9 

ERIC 


999 


Title: 


Self  Efficacy,  Performance  Variables  and  Distance  Learning 
Facilitator  Technology  Adoption:  Support  tor  the  Teacher  Needs  Hierarchy 


Authors: 

R.S.Talab 
BobNewhouse 


1001 


Innovation  and  Teacher  Technology  Adoption 
Seve.il  inv.atig.tors  have  pointed  out  that  the  success  of  reforms  which  rely  on  -outside" 
change  agents,  without  aufficiently  involving  teachers,  has  been  dismal  (Auat,  Bichelmeyer,  & 
Allen,  1991;Dwyer,  Wngsteff,  A  Sandholtz,  1990;  Janowitz  &  Street,  1966;  Johnaon  &  Keller, 
1981;  Martin  &  Clemente,  1990).  Cuban  (1986)  observed  that 

teacher  expertise  [is]  drawn  from  a  pool  of  craft  wisdom 
about  children  and  schooling  that  dances  beyond  the  limited 
understanding  of  non teaching  reformers...  (pp.  5^). 

Snyder  (1986).  citing  a  Rand  Corporation  study  of  school  innovations,  observed  that 
it  is  simplistic  to  wume  that  any  new  technology,  regardless  of  how  good  it  is,  w  be 
n>eanin8fu.ly  adopted  and  maintained  in  the  school  system  unless  it  take,  into  consideration  the 
•ocW  and  politic*  climate  of  the  school  and  places  the  teacher  'at  the  "deed  center  of  the  loop". 
'Historically,  teachers  use  technologies  that  buttress,  rather  than  undermine,  their  authority 
(Bichelmeyer,  1991). 

Investigators  have  found  mat  teachers  adopt  new  technological  innovations  around  a 
cluster  of  factors:  simplicity,  durability,  reliability,  versatility  (Johnson  &  Keller,  1981;  Rogers, 
1983),  flexibility,  time,  communication  (Kell,  et.  .1.,  1990),  and  developmental  input  (Aust,  et. 

1989).  Kell  and  others  (1990)  reinforced  the*  findings  by  naming  five  conditions  that  are 
conducive  to  change  in  the  classroom:  .)  a  shared  vision  of  teaching  and  learning,  2)  leadership 
«d  support  for  new  technology,  3)  organization,,,  conditions  allowing  flexibility,  time,  and 
incentives.  4)  peer  networking,  and  5)  training  and  personalized  support. 

Bichelmeyer  (199.)  expand*  on  these  factors  by  proposing  that  teachers  adopt 
technology  innovations  in  .  hierarchy  of  needs  based  on  Mallow'.  Hierarchy  (1968).  with  the 
most  basic  needs  generally  fc,lng  mm  Mon  higher  ones:  1)  time  and  accessibility.  2) 
d'pend.bil.iy,  3)  ownsrahlp  and  authority,  4)  control,  and  5)  integration. 
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Maslow  Biciulmeyer 

5  Self-Actualization  ..  *  Integration 

A 

4  Esteem  /    \  4  influence  on  Design 

3  Belongingness  .  \  3  Ownership  &  Authority 

2  safety  \       2  Equipment  Dependability 

1  Physiological      tl  \    1  Time  and  Equipment 

Accessibility 

Figure  1:  Hierarchy  of  Needs  and  Hierarchy  of  Teacher  Technology 
Needs 


Teacher  Needs,  Empowerment,  and  Technology  Integration 
This  model  is  based  on  empowerment  and  accounts  for  1)  the  centrism  of  the  teacher,  2)  the 
politics  of  the  school,  and  3)  the  practical  logistics  of  classroom  technology  incorporation 
(Jackson,  1968;  McDonald,  J.  1989). 

"...Technological  innovations  that  have  been  embraced 
by  teachers  are  those  that  have  solved  problems  which 
teachers  themselves  identified  as  important,  regardless 
of  what  non-teachers  say"  (Bichelmeyer,  1991). 
No  other  model  of  teacher  technology  adoption  accounts  for  all  of  the  above  three  factors. 
Self-Efficacy  and  Technology  Adoption 
Central  to  the  concepts  of  ownership  and  authority  is  "aelf-efficacy".  Bandura's  (1977)  theory 
states  that  people  develop  beliefs  concerning  their  own  coping  capabilities.  The  extent  to  which 
a  belief  is  internalized  by  the  teacher  affect*  the  value  of  an  endeavor  to  that  teacher.  Teachers 
that  internalize  or  vest  a  concept  thereby  increase  their  effectiveneea  in  ita  use  (Brophy,  1979; 
Gibson  &  Dembo,  1984).  Teachers  who  believe  that  they  have  succwfully  integrated  new 
technology  tend  to  be  teachers  who  successfully  integrate  technology  into  their  instruction  (Riggi, 
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1988).  "If  I  master  it,  then  I  can  internalize  it*  (Bichelmeyer,  1991,  p.  138), 
The  Distance  Facilitator  as  Technology  Adopter 

Satellite-based  instruction  is  an  instructional  delivery  mode  serving  over  125,000  students 
in  45  states  through  the  S.T.A.R.  Schools  program  alone  (Office  of  Educational  Research  and 
Improvement,  1992),  as  well  as  other  other  independent  and  consortium  providers,  such  as  The 
Kansas  Regents  Educational  Communications  Center,  TIE-IN,  SERC,  etc.  While  the  distance 
facilitator's  role  varies  somewhat  worldwide  (Harry,  1982;  Parer,  1990;  UNESCO,  1987),  in  the 
United  States  the  term  "distance  facilitator"  means  anyone  who  facilitates  distance  learning- 
certified  or  uncertified  and  regardless  of  their  duties.  The  "Teaching  Partner"  is  generally  a 
certified  teacher  who  must  learn  a  subject  along  with  the  students,  provide  course  and  equipment 
support  for  a  high  school  distance  class,  and  often  must  learn  along  with  the  students. 

The  distance  facilitator  plays  a  crucial  role  in  student  recruitment,  retention,  and 
persistence  (Hobbs,  1990;  Laube,  1992).  Distance  facilitators  are  considered  necessary  for 
effective  foreign  language  instruction  (Grier  &  Nelson,  1990).  Research  and  evaluation  data  on 
distance  facilitators  in  the  United  States  have  found  that  they  have  an  average  of  four  other 
preparations,  are  mid-career,  are  selected  by  their  principals  (rather  than  voluntarily  asking  to  be 
assigned  as  facilitators),  and  are  anxio  js  about  using  new  technology,  such  as  satellite  receivers, 
computers,  and  data  streaming  equipment  (Dillon,  1990;  Ford.  1990;  Hobbs,  1990).  Principals' 
reasons  for  facilitator  selection  are  based  on  subject  background,  availability,  and  general  teaching 
ability  (Talab  &  Newhouse,  1990). 

Research  indicates  that  teachers  and  instructional  designers  are  involved  in  similar  basic 
activities  (Applefield  &  Earle.  1990;  Branch,  Darwazeh  &  El-Hindi,  1992;  Gagne,  Briggs  * 
Wager,  1992).  Earle  (1992)  found  that  teachers  believed  that  a  knowledge  of  instructional  design 
processes  improved  their  planning.  It  is  likely  that  these  same  processes  may  be  internalized  and 
thereby  integrated  through  the  facilitation  of  satellite  instruction.  For  example,  course  elements 
include  the  regular  observation  of  a  "master  teacher,"  training  in  the  use  of  carefully  selected 
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print,  non-print,  and  instructional  rn.teri.ls  (syllabi,  teats,  worksheets,  etc.).  bardw«e  trdning. 
and  the  chance  to  network  with  other  facilitator,  on  coune  progrew  (Talab,  1991.).  Typical 
commenu  of  distance  ftcilitators  .bout  the  te«bingflearning  experience  are  th*  they  h.ve  gained 
from  le.rn.ng  from  an  excellent  teacher  end  Out  they  professionally  benefited  from  the  opportunity 
to  network  with  other  teachers  on  methodology  (Dillon,  1990). 

Purpose  of  the  Study 
The  purpose  of  the  present  study  is  to  examine  .elf-efficacy  and  performance  variables 
and  technology  adoption  for  secondary  disUnce  learning  facilitators.  Secondary  di.Unce 
facilitators  are  .  group  that  must,  by  the  definition  of  satellite-based  education,  adopt  the  most 
recent  technological  advances  available  for  instruction^  development  and  delivery.  There  are 
many  differences  between  adult  distance  learning  and  learners  and  secondary  distance  learning  and 
learners  (Laube,  1992).  To  date,  these  issues  have  yet  to  be  addressed. 

Hypotheses 

In  order  to  find  if  any  relationships  existed  between  certain  ownership/self-efficacy 
("vesting"),  performance  variables  and  technology  adoption  the  following  hypotheses  were 
developed: 

1.  A  high  vester  will  be  more  committed  to  continuing  as  a  distance  facilitator  than  the 
low  vester. 

2.  A  high  vester  will  perceive  mm/herself  as  a  more  effective  distance  facilitator  than 

the  low  vester. 

3.  A  high  vester  will  feel  more  comfortable  with 
with  new  technology  than  the  low  vester. 

4.  A  high  vester  will  be  perceived  by  the  principal  as  being  .  more  effective  disUnce 

facilitator  than  the  low  vester. 
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Methodology 

Subjects  and  Setting 

The  subject*  were  107  high  school  teachers  who  served  as  distance  facilitators  for  an 
introductor  Spanish  course  offered  by  the  Regents  Educational  CommunicaUons  Center  (RECC) 
at  Kansas  State  University.  The  course  is  offered  in  nine  states:  Alabama,  Colorado,  Kansas, 
Missouri,  Mississippi,  Montana,  Oklahoma,  Tennessee,  and  West  Virginia.  The  RECC  (and 
Kansas  law)  require,  the  presence  of  a  distance  facilitator  (-teaching  partner")  for  satellite-baaed 
courees.  This  person  must  be  a  certified  teacher  and  the  RECC  require,  that  he/she  learn  Spanish 
along  with  the  students.  Training  is  not  required  for  Teaching  Partners,  but  most  do  talc  part  in 
training  and  some  come  year  after  year.  During  training  discussion  groups  for  new  (Spanish  One) 
and  experienced  (Spanish  Two)  Teaching  Partners  were  formed  and  surveys  were  conducted  for 
formative  evaluation  purposes. 
Variables 

Variables  for  the  Teaching  Partner  survey  were  based  on  Bichelmeyer's  (1991)  Hierarchy  of 
Needs  and  Bandura's  (1977)  concept  of  self-efficacy.  Self-*fficacy  refers  to  the  internalization 
of  a  practice  or  principle  and  corresponds  to  the  ownership  and  authority  (3rd  level)  of  the 
Teacher  Needs  Hierarchy.  In  MasloWs  Hierarchy  of  Needs  level  3  would  be  Belongingness. 

The  performance  variable  chosen  was  instructional  design.  If  teachers  felt  that  as  a  result  of 
their  being  distance  facilitators  they  could  design  instruction  with  new  technology  then  they  would 
likely  exhibit  more  effective  teaching  performance  with  technology. 

Instructional  Design.  They  were  asked  if,  as  a  result  of 

being  a  Teaching  Partner,  they  knew  the  steps  necessary 

to  using  technology  in  the  instructional  setting. 

Three  self-*fficacy  variables  were  tested  against  the  performance  variable. 
Teaching  Partner  Commitment.  They  were  asked  if  they 
would  like  to  continue  as  a  Teaching  Partner. 
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Responsiveness  to  New  Technology.  They  were  asked  if 
they  were  more  comfortable  with  using  new  technology 
at  a  result  of  being  a  Teaching  Partner. 
Role  Effectiveness.  They  were  asked  if  they  were  performing 
their  duties  well. 

Instrumentation  * 

In  order  to  assess  the  relationship  between  ownership/self-efficacy  (vesting)  variables  and 
performance  an  81-item  questionnaire  was  developed  for  Teaching  Partners.  The  response  format 
to  questionnaire  items  varied  and  included  items  developed  on  a  five,  four,  and  three  point  Likert 
scale,  multiple  choice,  and  open  ended  response  items.  Eleven  of  the  questions  were  adapted  from 
Riggs'  Science  Self-Efficacy  Instrument  on  Microcomputers  (Riggs,  1988).  The  reliability  of  the 
performance  variable  was  .82  and  the  reliability  of  the  self-efficacy  variables  were:  Variable  1 
(commitment): .72,  variable  2  (technology  adoption): .81,  variable  3  (role  effectiveness):. 75 

A  thirteen-item  phone  survey  was  constructed  for  Principals  consisting  of  five  point  Likert, 
multiple  choice,  and  open  ended  response  items.  In  order  to  clarify  survey  questions  an  open- 
ended  interview  was  conducted  with  sixteen  Spanish  One  Teaching  Partners  participating  in 

training. 
Procedures 

Questionnaires  with  stamped  return  envelopes  were  mailed  in  May,  1991,  to  all  107  distance 
facilitators.  After  two  mail  followups  in  July  and  August,  89  questionnaires  were  returned  by  late 
September,  resulting  in  an  81  %  response  rate.  Phone  surveys  were  conducted  with  32  principals 
in  late  May  and  early  June,  constituting  a  one  in  five  sample.  After  dropouts  29  facilitator- 
principals  pairings  were  available  for  analysis. 
Statistical  Analyses 

The  first  three  hypotheses  were  tested  by  use  of  Chi  Square  analyses  (with  Yates  Correction) 
comparing  subjects'  self-efficacy  to  performance  variables.  This  was  accomplished  by  dropping 
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the  •undecided"  group  from  the  5-part  Likert  scale  and  dividing  the  remaining  groups  into  either 
high  or  low.  The  last  hypothesis  wis  tested  using  the  Pearson  Product-Moment  Correlation  to  see 
if  a  relationship  existed  between  self  efficacy  and  principals'  perception  of  Teaching  Partner 
performance.  Responses  from  the  open-ended  interview  with  Spanish  One  Teaching  Partners  were 
examined  using  a  content  analysis. 
Hypothesis  One 

The  results  of  the  Chi  Square  test  were  significant  (X2-6.54,  df- 1,  p<  .01)  indicating  that 
the  high  vester  group  differed  significantly  from  the  low  vester  group  in  their  commitment  to 
continuing  as  a  Teaching  Partner  (see  Table  I). 

Table  I  =  Comparison  of  Self-Efficacy  and  Commitment 


Commitment 


Group 

No 

Yes 

Total 

Vesters 

•0- 

26 

26 

Non-Vesters 

3 

5 

8 

Total 

3 

31 

34 

X2=6.54       df-1  p.<.01 
Miuing  Observations:  21 

Hypothesis  Two 

The  results  of  the  Chi  Square  test  were  significant  (X2= 15.29,  df=l,  p<.0001) 
indicating  that  as  a  result  of  being  Teaching  Partners  they  feel  comfortable  with  new  technology. 
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Table  2  -  Comparison  of  Self -Efficacy  and  Technology  Adoption 
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Technology 

Adoption 

Group 

No 

Yes 

Total 

V  esters 

-0- 

36 

36 

Non-Vesters 

3 

2 

5 

Tottl 

3 

38 

41 

X2=  15.29  df=l  p<.0001 
Missing  observations  =  14 


Hypothesis  Three 

The  results  of  the  Chi  Square  test  were 
that  the  high  vester  group  differed  significantly 
Table  3  =  Comparison  of  Self-Efficacy  and 


significant  (X2-3.99,  df»  I,  p<  .04)  indicating 
from  the  low  vester  group  in  their  role  attitudes. 
Role  Effectiveness 


Role  Effectiveness 


Croup 

No 

Yes 

Total 

Vesters 

1 

35 

36 

Non-Vesters 

1 

-0- 

1 

Total 

2 

35 

37 

X2-3.99     df=l  p<-.04 
Missing  Observations  =  18 
Hypothesis  Four 


The  data  were  inconclusive.  A  Pearson  Product-Moment  Correlation  could  not  be  done  due 
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the  absence  of  comparable  ceJU .  Most  principals  rated  the  quality  of  the  facilitator**  performance 
highly,  answering  on  a  five-part  Likert  acale  either  "excellent-  (14)  or  "very  good"  (13)f  for  a 
total  of  27  that  were  rated  highly.  One  principal  was  alio  a  facilitator  and  could  not  be  included. 
One  facilitator  was  judged  to  be  'very  poor'  but  responded  in  high  vaster  group  in  all  previous 
measures.  No  explanation  was  offered  by  the  Principal  for  the  poor  evaluation.  In  addition, 
principals  may  have  answered  more  positively  in  a  phone  interview  than  by  mail,  as  principals  will 
respond  more  positively  to  questions  from  interviewers  from  the  RECC  host  campus  phone  survey 
for  which  they  might  be  identified  than  those  for  which  remain  anonymous  (Talab,  1991b). 
Content  Analysis  of  Facilitator  Responses  to  Open-Ended  Survey 

Subjects  were  asked  to  identify  what  concerns  they  had  as  Spanish  One  facilitators.  A  total 
of  16  subjects  provided  responses,  and  most  gave  multiple  responses  for  a  total  of  34  responses. 
A  conknt  analysis  was  conducted  in  order  to  aggregate  similar  responses  into  categories. 
Facilitator  responses  are  ranked  in  order  of  frequency  of  response  for  each  category.  The  results 
were  two  areas  of  concern:  1)  instructional  design  (Table  5)  and  2)  classroom  management  (Table 
6). 
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Table  5:  Sanmary  of  Content  Analysis  of  Facilitator  Instructional 
Design  Concerns 
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Identified  Concerns  Frequency  of  Responses*  Rank 

difference!  between  book  emphasis  6  1 

and  tv  professor  emphasis 
course  element  timing  during  school  year    6  1 

the  need  for  an  anticipatory  set  5  2 

problems  with  question  cards  4  3 

should  students  write  or  speak  first  4  3 

coordinating  activities  3  4 


who  speaks  when  on  camera 


3  4 


♦Total  number  of  subjects  responding  =  16 

Table  6:  Summary  of  Content  Analysis  of  Classroom  Management 


Identified  Concerns  Frequency  of  Responses  Rank 

School  Schedules  9  1 

planning  time  to  work  on  materials  7  2 
difference  between  facilitating 

and  traditional  teaching  6  3 

need  for  in-class  time  for  student  work  5  4 

varied  responsibilities  3  5 

♦Total  number  of  subjects  responding  *  16 
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Other  statements  of  interest  made  by  facilitators  were  that  satellite-based  instruction  made  the 
students  responsible,  that  they  learned  to  become  better  learners,  and  that  other  teachers  were 
skeptics!  of  facilitators  because  it  looked  like  it  was  easy.  Most  (14)  participants  agreed  that  it 
was  just  as  difficult  as  traditional  teaching.  Two  participants  believed  mat  it  was  more  difficult 
because  of  the  management  and  technological  demands.  * 

Conclusions 

Among  the  three  self-efficacy  variables  tested  for  high  and  low  vesters  against  the 
performance  variable,  there  was  a  significant  positive  relationship  found  between  instructional 
design  and  1)  commitment,  2)  responsiveness  to  new  technology,  and  3)  role  effectiveness.  No 
statistically  significant  relationship  was  found  between  principals*  evaluation  of  facilitator 
performance  and  facilitators'  self-appraisal. 
Commitment 

The  results  of  this  study  support  the  findings  of  others  (Bichelmeyer,  1991;  Dillon,  1990; 
Reezabek,  1991;  Talab  &  Newhouse,  1990).  Mastering  the  technology  for  instruction  was  in 
large  part  a  reason  for  facilitators  to  continue.  It  must  be  noted  that  facilitators  generally  do  not 
volunteer  and  are  hesitant  about  using  technology  (Hobbs  &  Osbourn,  1989).  However, 
commitment  to  continue  seems  to  be  related  to  a  large  extent  on  program  training  and  to  a  lesser 
extent  on  program  implementation  rather  than  the  mere  adoption  of  the  technology  or  the  position 
itself  (Ford,  1990). 
Technology  Adoption 

The  finding  that  facilitators  feel  more  comfortable  with  adopting  new  technology  as  a  result 
of  being  a  facilitator  corroborates  earlier  research  that  presents  a  surprising  consensus  on  faciliator 
duties,  with  one  of  the  major  duties  being  equipment  operation  (Simonson,  Johnson  &  Neuberger, 
1989;  Mihalevich,  1990;  Talab,  1990a).  Kell  et.  al.'s  finding  (1991)  that  familiarization  with  the 
many  forms  of  media  and  technology  that  are  used  in  an  exemplary  class  is  also  meaningful  with 
regard  to  facilitator  exposure  to  technology  adoption  through  combined  with  training  and  modeling 
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by  the  muter  satellite  teacher.  The  one  •caveat"  is  that  technology  training  must  be  executed 
properly  and  technology  implementation  must  be  aided  by  communication  with  course  personnel, 
since  facilitator-perceived  weaknesses  with  satellite-based  instruction  training  center  on  technology 
and  equipment  training  (Ford,  1990). 
Role  Effectiveness 

The  findings  support  those  of  Hobbs  (1989)  and  Ford  (1990)  because  of  the  high  degree  of 
instructional  technology  that  must  be  mastered  in  order  to  fulfill  the  role.  It  seems  evident  that 
those  who  do  feel  that  they  have  a  high  degree  of  self-efficacy  in  instructional  design  with 
technology  would  feel  that  they  fulfill  their  role  to  a  high  degree  (Reezabek,  1991). 
Principals'  Perceptions  of  Facilitator  Role  Effectiveness 

The  inconclusive  findings  could  be  indicative  of  either  a  high  approval  rating  or  an 
unwillingness  on  the  part  of  principals  to  appear  that  satellite-based  education  is  anything  other 
than  successful  (Talab,  1990b).  A  feeling  that  is  generally  shared  by  school  adminstrators,  more 
so  than  facilitators,  is  that  satellite-based  instruction  is  an  effective  and  inexpensive  way  to  bring 
college  preparatory  instruction  to  rural  and  underserved  areas  and  this  could  account  for  a 
generally  positive  response  (Hobbs,  1990). 

Discussion 

The  Importance  of  the  Teacher  Needs  Hierarchy  in 
School  Restructuring 

Support  for  this  model  can  be  drawn  from  the  concept  of  lelfcfficecy.  High  vesters  are 
different  from  low  vesters  in  their  commitment,  role  effectiveness,  and  responsiveness  to  new 
technology.  They  believe  in  their  own  abilities  to  design  instruction  using  new  technology  as  a 
result  of  being  distance  facilitators.  In  many  ways  they  are  representative  of  all  teachers  in  their 
hesitance  to  learn  the  technology  and  the  fact  that  they  are  generally  selected  to  be  facilitators 
rather  than  volunteering  themselves.  Yet  they  believe  that  as  a  result  of  being  teaching  partners 
they  are  now  able  and  willing  to  continue  in  this  role.  By  all  accounts,  distance  facilitators  have 
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successfully  incorporated  technology  into  their  classrooms  in  an  exemplary  manner. 

The  question  then  becomes  "why  are  the  results  so  positive  with  satellite-baaed  education  in 
contrast  to  computer  integration?"  There  could  be  several  reasons.  Studies  show  that  teachers 
who  become  facilitators  are  no  more  conversant  or  willing  to  use  satellite  technology  than  they 
art  to  use  computers  (Hobbs,  1990;  Reezabek,  1991;  Talab  &  Ncwhouse,  1990).  Yet  most 
facilitators: 

1)  are  committed  to  the  concept  of  equality  of  education  (college  preparatory  courses  for 
underserved  students)  that  satellite-based  education  provides, 

2)  see  opportunities  for  professions!  advancement  through 
learning  new  skills  and  professional  networking; 

3)  seem  revitalized  by  the  observstion  of  a  master  teacher  and  exceptional  instructional  design, 

4)  realize  that  the  program  will  not  work  without  their  participation, 

5)  receive  training  in  satellite-based  instruction,  either  live  or  on  tape,  professions! 
troubleshooting,  and  program  feedback. 

Theae  findings  also  corroborate  Bichelmeyer's  Hierarchy  of  Teacher  Needs  because  teachers 
•ra  given  time  and  accessible  equipment  (level  1),  program  personnel  help  them  with  machine 
operation  and  troubleshooting  (level  2),  they  take  part  in  training,  program  planning,  and  control 
the  grading,  classroom  management,  and  classroom  activities  (level  3),  they  influence  the  program 
through  feedback  (level  4).  and  they  see  the  need  for  technology  integration  in  order  to  take  part 
in  the  program  (level  5). 

While  there  is  no  research  on  the  differences  between  facilitators  and  other  teachers,  a  possible 
answer  could  be  that  the  difference  is  philosophical.  Teachers  must  indeed  "internalize"  the  use 
of  a  new  technology  if  the  use  is  to  be  long-term  and  involve  more  than  just  the  standard  "top- 
down  "-principal-teacher  authority  structure  that  has  operated  in  traditional  education. 
Restructuring  with  technology  must  take  into  account  the  teacher's:  1)  centrism,  2)  authority  base, 
and  3)  design  involvement.   Ravitch  (1993)  states: 
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School  organization  has  been  traditionally  hierar- 
chical, bureaucratic... .New  technologies  challenge 
Ibis  model. 

We  mutt  recognize  that  one  of  the  main  reasons  that  new  technologies  have  not  been 
incorporated  into  the  schools  after  their  initial  introduction  il  that  the  teachers  have  not  been 
recognized  aa  motivating  force,  but  as  workers.  Teachers  are  the  instructional  leaders  of  their 
classrooms,  yet  their  importance  aa  individual  authorities  hss  been  ignored,  in  the  planning, 
introduction,  and  execution  of  many  new  programs  (Cicchelli,  1982;  Lipsitz,  1989).  Meaningful 
adoption  based  on  what  is  sure  to  be  constant  change  requires  the  use  of  a  new  model. 
Technology  Adoption:  Is  it  Exemplary  in  Distance  Education? 

There  sre  five  major  issues  in  distance  education:  1)  a  critical/reflective  framework  for  the 
field,  2)  access  and  equity,  3)  dislog  snd  independence,  4)  technology,  and  5)  third  world 
development  (Evans  &  King,  1991).  Distance  learning  technology  changes  rapidly,  requiring 
constant  adaptation  (Pelton,  1990).  Technology  adoption  is  a  key  issue  in  distsnce  facilitation 
because  it  is  technology-dependent.  Perhaps  the  issues  and  theory  base  that  undergird  distance 
education  snd  make  '  successful  in  the  introduction  of  technology  needs  to  be  examined  in  school 
technology  integration. 

The  Need  for  a  Pre-College  Theory  Base  of  Distance  Education 

The  fast-growing  use  of  satellite-based  instruction,  with  another  four  consortiums  being 

awarded  a  total  of  $18.2  million  for  the  next  two  years  (Electronic  Uaming,  1993),  brings  to 
10  the  number  of  multi-institutional  consortiums  that  have  received  Star  School  funding.  The  need 
to  look  closely  at  the  unique  characteristics  of  high  school  distance  education  United  States  and 
Canada  is  now. 

The  research  of  Laube  (1992)  and  others  (Dillon.  1992;  Speth,  1992;  Speth,  Poggio,  & 
Glasnapj  i992)  working  primarily  with  secondary  distsnce  education  in  the  United  States  and 
Canada  clearly  show  'no  consistent  trend  with  the  findings... found  in  adult  education"  (Uube, 
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1992).  The  operating  assumptions  we  different.  Secondary  students  have  different  motivations, 
controls,  and  restrictions  than  do  adult  students.  Secondary  distance  facilitators  have  different 
time  and  professional  constraints,  as  well.  With  the  use  of  distance  education  around  the  world 
in  the  pre-college  setting  and  the  rapidly  growing  population  of  United  States  and  Canadian 
secondary  distance  learning  it  is  essential  that  a  theory  base  be  constructed  in  order  to  better  serve 
this  constituency. 
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Abstract 

We  investigated  the  effects  of  studying  alone  or  in  cooperative-learning  groups  on  the 
performance  of  high  and  low  achievers.  We  also  examined  the  effects  of  completing  computer- 
based  instruction  using  either  learner-  or  program-control.  A  total  of  92  sixth-grade  students  were 
classified  by  Stanford  Achievement  Scores  and  randomly  assigned  to  group  or  individual 
treatments,  stratified  by  achievement  scores.  Students  completed  training  to  enhance  small  group 
interaction  before  completing  the  computer-based  tutorial  and  post-tests.  Both  high  and  low 
achievers  in  the  cooperative  treatment  increased  achievement  on  program-controlled  and  learner- 
controlled  computer  lessons.  The  learner  controlled  cooperative  learning  group  made  more  options 
while  checking  their  concept  learning,  and  spent  more  time  interacting  with  the  learner-controlled 
computer-based  tutorial,  than  the  ieamer-controlled  individual  learning  group. 
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The  Effects  of  Cooperative  Learning  and  Learner  Control  on  High-  and  Low  Achievers 

Designers  of  instruction  agree  that  it  is  important  to  identify  how  the  instructional 
environment  can  be  organized  in  order  to  utilize  the  relevant  cognitive  processing  capabilities  of 
learners,  and  to  examine  the  manner  in  which  technology  facilitates  such  processing  (Gagne, 
1987)  One  strategy,  which  is  the  subject  of  debate,  is  the  effect  of  the  shared  learning  procedures 
that  occur  when  students  work  in  groups  in  a  mediated  instructional  environment  using  modern 
technological  media  (Hooper  &  Hannafin,  1988).  Although  research  on  cooperative  learning  has 
been  conducted  to  maximize  the  cost-effectiveness  of  existing  media,  it  is  beginning  to  reveal  the 
cognitive  processing  potential  when  students  are  allowed  to  make  joint  decisions  regarding  their 
instructional  needs  (Johnson  &  Johnson,  1989). 

To  date,  there  arc  few  studies  that  have  focused  on  cooperative  learning  groups  with 
computer-based  instruction,  which  enable  learners  to  work  in  program-controlled  situations. 
Another  area  of  research,  which  has  become  popular  with  the  increasing  use  of  computers  in 
education,  is  that  related  to  delegating  some  instructional  decision-making  to  learners.  Using  the 
capabilities  of  computers,  it  is  possible  to  incorporate  several  types  of  control  strategy  in  the 
instruction. 

The  purpose  of  this  study  is  to  examine  two  factors  that  influence  the  effectiveness  of 
computer-based  instruction  (CM)  on  promoting  learning:  cooperative  group  learning  and  learner 
control  We  propose  that  efficient  use  of  observation  and  peer  modeling  (Bandura,  1977)  are 
powerful  mediators  of  improved  performance.  However,  younger  or  less-skilled  learners  are  often 
lacking  in  knowledge  of  the  use  of  strategies,  as  well  as  how  to  monitor  strategy  use  when  given 
the  opportunity  to  do  so.  In  cooperative  learning  groups  that  are  heterogeneous  with  respect  to 
ability,  low-  and  high-ability  students  will  have  an  opportunity  to  observe  each  other's  opuon- 
selection  skills.  This  research  will  investigate  whether  students  in  cooperative  learning  groups  wall 
help  each  other  monitor  the  skills  required  for  learner-controlled  behavior  and  improved  decision- 
making, while  learning  from  a  computer-based  instructional  environment  The  present  study  is  an 
attempt  to  explore  the  cognitive  and  metacognitive  assumptions  underlying  choice  behavior  and 
performance  in  a  learner-controlled  environment.  In  the  current  research  we  hypothesize  that 
children's  knowledge,  use,  and  monitoring  of  cognitive  strategies-especially  those  related  to  the 
decisions  made  during  computer-based  instruction-may  be  inefficient,  resulting  in  poor 
performance  and  non-strategic  use  of  available  options.  For  example,  it  is  not  sufficient  to  merely 
provide  learners  with  options  in  a  lesson,  without  guidance  on  how  to  use  them  wisely  (Boyd, 
Douglas  &  Lebel,  1984;  Tobias,  1987).  Learner  control  can  be  considered  a  meta-strategy,  in 
which  learners  must  exercise  a  high  level  of  self-monitoring. 
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One  of  the  goals  of  instruction  is  to  promote  independent,  life-long  learning.  This  goal 
may  be  promoted  by  encouraging  students  to  exercise  learner  control.  Learner  control  tufers  to 
students*  freedom  to  manipulate  the  instructional  sequence.  In  CBI,  designers  may  allow  students 
considerable  freedom  (or  they  may  impose  rigid  control)  to  choose  the  number  of  examples  and 
questions,  which  instructional  components  to  complete,  and  when  to  quit  instruction. 

A  large  body  of  research  suggests  that  students  often  perform  poorly  in  learner-controlled 
conditions  (Steinberg,  1989).  However,  this  effect  is  mediated  by  achievement  High  achievers 
appear  to  exercise  effective  learner  control,  but  low  achievers  perform  particularly  poorly  under 
such  conditions.  Low  achievers  often  lack  sufficient  self-regulatory  strategies  to  make  effective 
learner-controlled  decisions. 

The  effect  of  peer  interaction  on  students'  proficiency  in  making  effective  decisions  has  not 
been  examined.  Yet,  related  research  suggests  that  low  achievers  may  work  effectively  in  learner- 
controlled  conditions,  when  paired  with  a  more  capable  partner.  High-achieving  students  often 
model  effective  learning  strategies  to  less  capable  partners  (Bandura,  1977).  We  predict  that 
students  in  the  learner-controlled  conditions  will  learn  more  effectively  in  groups  than  with 
individual  treatments. 

The  purpose  of  this  study  was  to  extend  previous  research  comparing  the  effects  of 
completing  CBI  in  groups  and  alone.  The  study  examined  the  effects  of  cooperative  versus 
individual  CBI  on  the  performance  of  high  and  low  achievers.  The  study  also  examined  the  effects 
of  learner  and  program  control  in  individual  and  cooperative  treatment  Finally,  the  study 
investigated  students*  option  within  learner  controlled  group  to  see  the  effects  of  time  and  number 
of  options  on  individual  and  cooperative  treatments. 

Method 

Subject 

A  sample  of  92  sixth-grade  students  from  a  predominantly  white  suburban  school, 
participated  in  the  study.  Participation  was  voluntary,  and  only  those  who  had  parental  consent 
were  included.  The  standardized  reading  scores  of  this  sample  were  obtained  from  the  SAT 
(Stanford  Achievement  Test )  test  battery  administered  at  the  beginning  of  the  academic  year 
(Mcan=  48.01  ,Range=  2  to  95  ,SD=  26.95  ).  Students  werc  classified  as  high  or  low 
achievers  according  to  performance  on  the  reading  subscale  of  the  Stanford  Achievement  Test. 
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Materials 

The  following  materials  were  used  for  the  study. 
franrfarrii7i»d  scores 

Standardized  achievement  test  scores  in  reading  ability  and  overall  scores  were  collected  at 
the  beginning  of  academic  year  and  before  the  study.  Because  the  experimental  content  is  highly 
verbal  (textual)  in  nature,  these  scores  were  used  to  form  the  heterogeneous  learning  pairs. 
Reading  ability  has  been  found  to  predict  performance  in  a  learner-controlled  task  based  on  science 
content  (eg.,  Kinzie,  Berdel  and  Sullivan,  1988). 

Cmnp"t(>r-Rasftd  tutorial 

The  computer-based  tutorial  used  for  the  study  was  designed  and  developed  by  the 
experimenter  using  biological  concepts  that  were  used  in  another  study  on  learner-control  (Relan, 
1991).  Several  steps  were  involved  in  the  selection  of  content,  design,  and  development  of  the 
tutorial. 

.^l^tinn  of  content    The  topic  of  Ecology  was  selected  because  of  the  requirement  from 
the  principal  and  sixth-grade  teachers  in  this  elementary  school.  Content  was  selected  with  the 
objective  of  identifying  level  of  learning  outcomes  (Gagne\  Briggs,  and  Wager,  1988).  Definitions 
and  instances  of  ecological  topics  were  excerpted  from  biological  ecology  texts  ranging  from  sixth- 
grade  to  college-level  textbooks.  These  were  paraphrased  to  accommodate  the  text  to  the  target 
population  according  to  the  purpose  of  the  study. 

The  topic  selected  was  "Relationships  among  Organisms,"  which  included  two  types  of 
relation:  friendly  and  unfriendly.  Six  topics  were  identified  for  inclusion  in  the  lesson:  Mutualism, 
Commensalism,  Competition,  Parasitism,  Exploitation,  and  Predation. 

Post-test.       A  25-item  multiple-choice  test  on  generalization,  based  on  verbal  and 
inferential  learning  outcomes,  was  implemented  as  both  immediate  and  delayed  post-tests.  This 
test  was  used  in  previous  research  and  had  been  tested  for  reliability  (ct=  .79). 

rVpftnftent  Measures 

Three  measures  were  obtained  for  each  individual  student:  achievement,  time,  and  options. 
Achievement  was  obtained  through  the  immediate  and  delayed  post-tests.  Time  was  collected 
during  the  computer-based  tutorial.  Options  were  the  numbers  of  options  that  students  made 
during  the  tutorial  in  the  learner-controlled  treatment. 

Eesign         The  study  employed  two  designs.  For  individual  measures,  a  randomized  block 
design  was  employed  with  two  cross-experimental  factors,  Grouping  (individual  or  cooperative) 
and  Source  of  Control  (learner  or  program),  and  one  blocking  factor,  achievement  (high  or  low). 
Because  the  effects  within  treatment  levels  were  not  of  particular  a  priori  interest  (i.e.,  high 
achievers  were  always  expected  to  outperform  low  achievers  on  the  post-tests),  the  achievement  by 


1025 


Group  &  Control  5 


treatment  interaction  effect  was  partitioned  into  those  effects  of  more  interest  in  this  study.  That  is, 
the  effects  of  source  of  control  and  Grouping  were  examined  within  each  achievement  level.  The 
full  design  model  is  reflect  in  Table  1 .  For  learner  control-based  performances,  a  completely 
randomized  design  employed  with  two  crossed-experimental  factors:Grouping  (individual  or 
cooperative)  and  Level  of  Achievement  (high  or  low). 


Insert  Table  1  about  here 


Experimental  Procedures 

During  the  week  before  the  experiment,  students  completed  a  training  session.  Training 
lasted  approximately  50  minutes,  and  was  completed  in  an  intact  class. 

Subjects  were  assigned  to  treatments  using  stratified-ramdom  sampling.  Initially,  high-and 
low  achievers  were  randomly  assigned  to  paired-  and  individual-treatment  groups.  For  each  class, 
subjects  in  the  paired  treatment  were  ranked  within  each  achievement  level  group.  Partners  were 
assigned  by  combining  students  with  identical  ranks:  The  highest  achiever  was  paired  with  the  one 
who  scored  just  below  the  median;  the  second  highest  was  paired  with  the  subject  who  had  the 
second  score  below  the  median  and  so  on,  such  that  the  student  just  above  the  median  was  paired 
with  the  student  who  had  the  lowest  score.  In  this  way,  there  was  always  a  difference  between 
partners  of  at  least  1.5  standard  deviation  in  the  SAT  score,  and,  heterogeneity  among  group 
members  was  established.  Other  students  worked  individually.  The  immediate  post-test  was 
administered  on  the  same  day,  after  the  computer-based  tutorial.  One  week  following  the 
experiment  students  completed  the  delayed  post-test  individually.  Three  absentees  completed  the 
post-test  two  days  later.  Ninety-two  subjects  completed  the  study. 

Results 

The  results  are  presented  in  two  parts.  First,  results  are  reported  for  individuals. 
Reliabilities  and  inter-correlations  among  the  dependent  measures  are  presented,  followed  by 
separate  MANOVAs  examining  various  treatment  effects.  Subsequently,  results  are  reported  for 
each  of  two  sets  of  variables  on  individual  students:  immediate  post-test  and  delayed  post-test  The 
second  set  of  results  are  group-based  measures.  Group-based  measures  reflect  data  recorded  at 
each  computer.  Each  pair  of  students  in  the  cooperative  groups  produced  a  common  data  set  and  is 
thus  treated  as  one  case. 
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Calculations  were  made  using  the  Statistical  Package  for  the  Social  Sciences  (SPSS)  4.0 
for  Macintosh,  (@  1990,  SPSS  Inc.).  All  tests  of  significance  adopted  an  alpha  level  of  .05. 
Inconsistent  sample  sizes  reported  across  the  Tables  reflect  subjects  with, isolated  missing  data 
points. 

H'",H tSLflPa  and  Sate  ^liabilities       The  multiple  choice  test  on  generalization  was 
used  as  both  immediate  and  delayed  post-tests.  A  preliminary  correlational  analysis  was  performed 
to  examine  the  relationships  among  significant  variables  in  the  study.  Results  of  this  analysts  are 
shown  in  Table  2.  Table  2  presents  inter- correlations  among  all  the  immediate  post-test,  delay 
post-test,  reading  and  overall  scores  of  the  Stanford  Achievement  Test.  All  achievement  variables 
were  highly  correlated:  The  two  post-tests  were  significantly  correlated  and  were  also  related 
highly  with  both  reading  subscale  and  over  all  Stanford  achievement  scores. 


Insert  Table  2  about  here 


Table  3  reports  means,  standard  deviations,  and  alpha  coefficient  reliabilities  of  immediate 
and  delayed  post-test  The  immediate  post-test  reliability  was  .67.  The  delayed  post-test  reliability 
was  .73. 


Insert  Table  3  about  here 


Post-tests.      Table  4  reports  the  means  and  standard  deviations  of  immediate  and  delayed 
post-test  scores  for  treatment  effects  within  level  of  achievement  groups. 


Insert  Table  4  about  here 


The  immediate  post-test  shows  that  learner  control  cooperative  treatment  scores  ( M-l  2.75, 
SD  =  4  99)  were  higher  than  any  other  treatment  scores,  within  high  achievers.  However  program 
control  cooperative  treatment  scores  (M=8.91,  SD=  3.15)  were  higher  than  any  othei  treatments 
within  low  achievers. 
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The  delayed  post-test  also  shows  that  learner  control  cooperative  treatment  scores  were 
higher  ( M=  15.42,  SD=  4.34)  than  any  other  treatment  scores  within  high  achievers.  Program 
control  cooperative  treatment  scores  ( M=l  1.36,  SD=  2.29)  were  higher  than  any  other  treatments, 
within  low  achievers.  It  appears  that  the  high  achievers  performed  better  under  the  learner-control 
cooperative  treatment  than  the  program-control  cooperative  treatment  and  both  learner  and  program 
control  individual  treatments.  On  the  other  hand,  the  low  achievers  performed  better  under  the 
program-control  cooperative  treatment  than  the  learner-control  cooperative  treatment  and  both 
program  control  and  learner  control  individual  treatments.  However  there  arc  no  significant 
differences  for  the  control  main  effect. 

On  the  delayed  post-test,  the  MANOVA  indicated  significant  effects  for  grouping  within  the 
high  achievement  group  (F=l  1.57,  p<.001)  and  for  grouping  within  the  low  achievement  group 
(F=4.69,  p<  .05).  Both  high  and  low  achievers  learned  more  effectively  in  the  cooperative 
treatment  than  in  the  individual  treatment. 

Group  Results  (\  ^nffr-pontrnllpd  npfjnns) 

Table  5  reports  the  means  of  the  five  performance  indicators  during  the  learner-con  trolled 
lesson  across  Grouping  (Individual  and  Cooperative)  and  two  level  of  achievement  (High  and 
Low).  Leamer  control  cooperative  treatment  has  the  highest  means  of  Time  on  task  and  Checking 
the  concept  learning. 


Insert  Table  5  about  here 


A  MANOVA  on  the  five  performance  indicators  (time  on  task,  total  checking,  total  review, 
total  elaborative  feedback,  and  total  practice)  was  performed  to  compare  the  differences  in 
performances  during  learner  control  between  low  and  high  achievers  working  individually  and 
high  and  low  achievers  working  in  cooperative  pairs.  Results  of  the  MANOVA  indicated  that  there 
are  significant  differences  between  the  two  performance  indicators,  which  are  the  time  on  task  (df= 
2, 27,  F=8.82,  p=  .001)  and  the  number  of  learning  checks  for  each  concept  (df=  2,  27,  F=  4.08, 
p=  .028).  Univariate  follow  up  tests  indicated  that  high  and  low  achievers  working  together  in 
cooperative  treatment  pairs  spent  more  time  and  checked  their  learning  more  for  each  concept  than 
high  and  low  achievers  working  alone.  The  levels  of  significant  differences  are  shown  in  Table  6 


Insert  Table  6  about  here 
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Discussion 


The  purposes  of  this  study  were  to  examine  the  effects  of  completing  CBI  alone  and  m 
cooperative  learning  groups  on  high  and  low  achievers  and  ^^.^ 
program-controlled  treatments.  Cooperative  learning  improved  the  achrevement  of  both  h.gh  and 
low  achievers.  The  cooperative  learning  groups  demonstrated  higher  delayed  post-test 
achievement  than  the  individual  learning  groups. 

During  the  learner-contfolled  lesson,  cooperative  learning  groups  spent  more  time 
interacting  with  the  lesson;  they  also  checked  their  concept  learning  more  than  those  in  individual 
learning  groups.  We  interpret  these  findings  to  mean  that  cooperative  learning  mediates  deeper 
conteni processing  and  that  achievement  gains  are  the  .suit  of  greater  exploration  mtte  learning 
process  These  results  also  support  the  previous  study  (  Hooper,  Temiyakarn,  and  W.llrams, 
1991 ,  in  press).  This  is  important  because  it  challenges  the  notion  that  heterogeneous  cooperauve- 
leaming  groups  interfere  with  the  performance  of  high  achievers. 

However  the  learner  control  effect  is  mediated  by  achievement  (Carrier  and  Jonassen, 
1988)  Most  research  on  learner  control  suggests  that  high  achievers  appear  to  exercise  effecuve 
learner  control,  but  low  achiever,  perform  particularly  poorly  under  such  conditions.  Low 
achievers  often  lack  sufficient  self-regulator?  strategies  to  make  effective  learner-control  decis.om. 
However  the  effect  of  peer  interaction  on  students'  proficiency  in  making  effecuve  decisions  has 
not  been  examined.  This  study  suggests  that  low  achievers  may  work  effectively  in  learner- 
controlkd  conditions,  when  paired  with  a  more  capable  partner.  High-achieving  students  often 
model  effective  learning  strategies  to  less  capable  partners  (Bandura,  1977).  Thrs  study  shows  that 
low  achievers  in  the  learner-controlled  conditions  learned  as  effectively  as  in  the  program- 
controlled  condition  when  they  were  in  groups  than  when  they  were  in  individual  treatments. 

The  absence  of  significant  achievement  differences  between  program-control  and  learner- 
control  treatments  indicates  that  some  students  may  have  interacted  with  the  lesson  content 
superficially.  Previous  research  indicates  that  low  achievers  did  poorer  using  learner  control  than 
using  program  control  (Carrier  and  Jonassen,  1988).  However,  this  study  shows  no  differences 
between  program  control  and  learner  control  within  low  achievers.  Thus  learner  control  promoted 
the  bigger  mean  within  high  achievers. 

The  results  of  this  study  suggest  that  greater  attention  should  be  given  to  designing  CBI  for 
cooperative  learning  groups,  rather  than  individual  learning^  ^ 

promoted  by  cooperative  learning  provide  encouragement  to  stay  on  task  and  exert  memal  effort  to 
both  low  and  high  achievers.  In  the  cooperative  learning  environment,  high  and  low  achievers 
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explore  diverse  ideas  and  procedures  of  learning  (McDonald,  Dansereau,  Garland,  Holley,  & 
Collins,  1980).  Together,  computer-based  instruction,  learner  control,  and  cooperative  learning 
magnify  each  others'  benefits. 

Future  research  should  pay  strong  attention  to  using  the  powerful  learner-  controlled 
treatment  to  improve  achievement,  and  using  the  cooperative  learning  to  promote  learner  control, 
for  both  high  and  low  achievers.  Learner-controlled  lessons  should  be  carefully  designed  for 
computer-based  tutorials.  Learner  control  with  advisement  may  be  appropriate  to  adapt  the  student 
to  start  with  a  program-controlled  environment  and  gradually  move  to  a  completely  learner- 
controlled  lesson,  when  ththeir  learning  for  each  conceptey  are  ready  for  such  strategy. 
Modifications  of  this  experimental  design  may  enable  exploration  of  other  factors  that  enhance 
learning  within  the  learner-controlled  situation. 
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Table  1 

MANQVA  of  treatment  effects  on  the  post-tests  within  low  and  high  achiever  groups. 
Effect  df  F 


Immediate 

Blocking  factor 

Achievement  Level 

Treatments  within  Low  Achievers: 
Grouping 
Source  of  Control 
Grouping  by  Source  of  Control 

Treatments  within  High  Achievers: 
Grouping 
Source  of  Control 
Grouping  by  Source  of  Control 

Delay 

Blocking  factor 

Achievement  Level 

Treatments  within  I^w  Achievers- 
Grouping 
Source  of  Control 
Grouping  by  Source  of  Control 

Treatments  within  High  Achievers: 
Grouping 
Source  of  Control 
Grouping  by  Source  of  Control 


1,91  30.90  .000 


1,91  2.27  .136 

1,91  .07  .787 

1,91  .12  .729 


1,91  .83  .364 

i,  91  .64  .425 

1,91  1.39  .242 


1, 91  21.00  .000 


1,91  4.69  .033 

1,91  .98  .324 

1,91  .84  .363 


1,91  11.57  .001 

1,91  .06  .811 

1,91  1.43  .235 
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Table  2 

Correlations  between  independent  and  dependent  variables  of  interest  in  the  study. 


SAT 


Delay  Post-test  .7966 


frlote.  **  implies  p<.01. 


Immed.  Delay  SAT 

Post-test  Post-test 


SAT  Reading  i6539*^  -5766**  .8721 


.6711**  -6406 


** 
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Table  3 

Means,  standard  Deviation,  and  Reliabilities  for  the  achievement  Measures 
N=92 


Scale  Mean 

Standard  Deviation 

Coefficient  a 

Immediate 

10.03 

3.93 

.67 

Delay 

10.91 

4.34 

.73 

1  U3* 
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Table  4 

Mnm  ^  ctonHarH  ^  of  inwdiatr.  and  drlnv  tirnt  int  rmn  bv  tiwimH. 

Level  of  control  and  Grouping 
 PC 


Level  of 
Achievement 


Immediate 
N 

Means 
SD 

Low 
N 

Means 
SD 


LC 


10 

10.60 
3.95 


13 

7.69 
1.93 


12 

12.75 
4.99 


12 

8.83 
2.33 


11 

12.64 
3.67 


12 

7.08 

1.08 


Coop. 


11 

12.36 
4.57 


11 

8.91 
3.15 


Total 


44 

12.14 
4.28 


48 
8.10 
2.28 


Delay 

High 

N 

Means 
SD 

Low 
N 

Means 
SD 


10 

10.30 
3.40 


13 

8.00 
2.48 


12 

15.42 
4.34 


12 

9.33 
1.72 


11 

11.36 
4.99 


12 
8.08 
2.58 


11 

13.82 
5.90 


11 

11.36 
2.29 


44 

12.84 
5.03 


48 
9.13 
2.60 


Note.  Maximum  points  possible  =  25. 
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Table  5 

Uama^aiiEBliMatona 


Level  of  group 
within 

learner  control 


Individual 

Low  achievers 
N 
M 
SD 

High  achievers 
N 
M 
SD 

Cooperative 
High  &  Low 
N 
M 
SD 


Total 


N 
M 
SD 


Time 


10 

24.00 
5.23 

10 

23.60 
2.K4 


10 

31.20 
5.  IK 


30 

26.67 
5.05 


Check 


10 

4.10 
3.51 

10 

3.70 

3.34 


10 

7.20 
1.87 


30 
5.00 
3.30 


Ty|» 


'  Options 


Rev  *w 


v() 

.') 
') 

70 
X2 


■I 

.70 
06 


Ehiborative 
Feedback 


;o 

5.00 
4.83 

10 
5.30 
4.92 


10 

8.90 
4.95 


30 

6.40 

5.06 


Practice 


10 

22.20 
6.73 

10 

21.00 
4.92 


10 

20.60 
4.90 


30 

21.27 
5.43 
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Table  6 


oOgaimsal  rffrm  on  lbs  options  adlhin  rooprmtiYclcamiD^^ 


Variable 
Time  on  task 

Checking  Concept  Learning 
Review  Each  Concept 
Elaborative  Feedback 
Practice  Items 


df 

F 

P 

2,27 

8.82000 

.001 

2,27 

4.08450 

.028 

2,27 

.04128 

.960 

2,27 

1.95948 

.160 

2,27 

.22233 

.802 
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Faculty  Concerns  as  Gateways  to  Teacher 
Competency  with  Computer  Technologies 

Introduction  and  Background 


oft* [X£  I  JSS^P°°       ^ faCed  wrth  pr0Viding  """P*"*  experiences  by  faculty  who 
22  3       ^  COmput,nfl  experiences  and  insufficient  resources.  The  International  Societv  for 
Technology  ,n  Education  (.STE,  1992)  developed  standards  in  education*  . SnCnTapSSons 

SSS:  J^fe  standards  have  also  been  approved  as  m^mSSSSS^m 
p3  ff '  Accreditation  of  Teacher  Education,  thereby  increasing  the  pressure  on 

edUCati°na'  ^Ponents  within  Curses'  IZniSng 

p^ess       *  development  and  resource  acquisition  is  an  important  step  in  the  planning 

tea^S  in 
InngvafcnQi^ionnflirerSoCQ] (Hall.  George  &Ruthertord.  197^dSu^Sfi^erns in 
imegrating  computer-based  technologies  in  courses?  And  second  because  some  faeum/  hart 

ss^sssssr puter  activities  in  ^ what  ^^^s^sjsl 

tnose  wno  have  had  that  expenence  and  those  who  have  not? 

This  th^Sm^Sf  ^0Pti°r  MOde' ICBAMJ  P™"* the  conceptual  framework  for  this  study 
Tnismeory  assumes  that  change  Is  a  process  that  follows  a  seven  stage  devetoDmental  JauJZ 
regarding  the  concerns  that  faculty  have  when  an  innovation  is  a^pled  ^HaTS toT 

St^Slr  ^  ^ d"*°P™*<  nature  if  teaSnetSnS.  S^e 
btSm.^n^^^  f  ^'^P^^  with  seff  concerns.  As  student  teachers  they 
co^cZe?^^;^  1  m?2m9  th* tMkS  0f  teachin9-  Wrth  ™e  experience,  they  tecome 
sZen^ofco^en^  ^  and  <1973>  f0"nd  thatthesVSeT 

KhSimp^e^  ,mP8C,)  WSre  3180  Present  when  teachers  were 

fc!toS^2^^2^SS!!'  5 ^2711"  awareness  of  the  nat"e  of  the  innovation.  This  is 
FuterftSc VZSZi  iSSl  J"!  tf!ird  f9*  reflects  P6^31  concerns  «*»ut  the  innovation 
toSSStoSSl  are nSfJ'VJf  0UrthMsta9e' when  management  of  the  innovation  and 
SJ^IIJSl  ssuesJre  Pnorttie*  The  impact  concerns  were  expanded  to  three  staoes  The  fifth 
^n0Vat!°n  h"  ^  im^^  ^  a  concern  about  the  toJZ 

Strathe  (1985)  developed  inservice  for  computer  use  by  education  faculty. 

Methods 

had  SS^SS?  "tSXEXSTS  universrty  participated  in  this  **■ A" 

year  Some  ScXwSe  lEEZiSST1  P^ram  Certifies  an  avera9e  of  400  ^chers  each 
Lprogram^r^^  Students, 
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Data  Sources 


n*t*  w.«rp  aatharad  in  two  ways  First,  lacutty  completed  the  Slagg?  2l  P?n<?yn  Abpyrt  ?n 

^£SSS^^  to  v'lty  "'otive  intensities  ot  M*  of  «te  stages. 

Data  Analysis 

For  the  SoCQ  raw  scores  tor  each  of  the  seven  sub-scales  were  tallied  and  converted  to  norm* 
BAM  studies  have  indicaled  that  experience  with  an  .nnovaton  was  an  •  J^anUactor  irv me 

ssssrcsifsss-SK » — -°~  — — — - 

each  of  the  seven  stages. 

Raaulta  and  Discussion 

Figure  1  displays  a  line  chart  of  the  average  percentiles  lor  the  ^^J^.^^ 
oercentile  scores  lor  the  instsructional  users  and  non-users.  Thirteen  faculty  w«renc'a.^^"  urf rs 
rtS3n£5£t  That  is.  they  had  Incorporated  computing  activities  ^^J^S^^SSmi 
non-users,  who  had  yet  to  include  computer  components  m  courses.  Table  2  displays  the  distnouuon 
of  intensities  ot  concerns  between  the  two  groups. 


insert  Figure  1  and  Table  1  &  2  here 


Questionnatte  oats  revealed  thattaculty  as  a  whole  h*  M ^nw»  « & 

SSSS^  SSSS?!^*      "■"■» 01  °°ncerns 

""^tSS  SSSoS^L  (n-13)  had  more  intense  concerns  at  the  ^-ta.«- 
M  ISSESSTl*      ^  hShes,  intense  ^S^ffi^S^ 

SnSms  oShSS J*£ sell  concern  at  the  petsonal  level:  however,  this  person  also  hads^ond 

E£i wncamtor  impac.  concerns,  in  t^foWfSS,  5?  S  tor 
inexperienced  non-users  (n=1 3)  had  the  highesl  intensity  tor  sell  concerns,  particularly  tor 


1041  :o3o 


CBAM 


T«ble  1-  Perctntll*  Muns 


Sta9e  0         1         2         3         4         5  6 

Aware  Inform  Person  Manage  Conseq  Collab  Retocus 

Instructional 


Users,  n=l3 

42 

57 

52 

45 

69 

64 

68 

Non-users,  n=i3 

81 

83 

73 

52 

45 

43 

47 

Total:  All  Faculty 

61 

70 

62 

48 

57 

53 

58 

Distribution  of  SoCQ  Highest  Intensity  of  Concern 


Stage 

0 

Aware 

1 

Intorm 

2         3         4         5  6 
Person  Manage  Conseq  Collab  Refocus 

Instructional 
Users,  n=13 

1 

1 

2 

2 

2 

3 

2 

Non-users,  n=13 

7 

6 

0 

0 

0 

0 

0 

Total:  All  faculty 

8 

7 

2 

2 

2 

3 

2 

Percentage 

30.7 

26  9 

7.7 

7.7 

7.7 

11.5 

77 

V  «J 
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,ntormat.on  (percentile^),  and  were  least  concerned  about  collaboration  (percentile^).  All  of  the 
non-users  scores  revealed  highest  intensity  lor  awareness  and  information. 

Stages  ol  Concern 

Further  analysis  involved  examining  the  average  responses  to  eatf ieT toe 
scaies  The  toliowina  is  a  discussion  of  the  highest  rated  Item  for  each  of  the  sub-scales  ana 
mSSSSS^  of  faculty  about  the  most  trequently  mentioned  categories  of  concern 

°btaSeS  KoTtoe  awareness  sub-scale  indicated  two  genera,  ideas.  One  was  related  to 
knowledge  about  the  innovation  and  the  other  was  related  to  being  involved  with  other  issues, 
£S^tt»ta5fea  Among  the  non-users  who  had  highest  awareness  concemswerethoae 
EES  not  feel  that  they  were  competent  computer  users  as  well  as  those  who  are  more  concerned 
with  their  own  special  research  interests,  rather  than  technology. 

All*  toe  items  on  this  sub-scale  were  actually  rated  lowest  on  the  «fy^*23LJZ 
converted  to  percentile  scores,  even  a  low  rating  indicates  a  hfch  mtensrty  The : highest  ratednem 
«,a<;  that  of  beinq  "completely  occupied  with  other  things*  means:  all  f  aculty=2.8  on  the  0  7  «»e- 
nSrSv  8  noTusers=3  3)  One  non-user  spoke  of  having  too  much  to  teach  in  his  course  to  "spend 

a^we" as  others' ,acked  an  awarenr ot  waysto  f 

VSSL  SoTS  compiler  activities.  An  experienced  user  with  highest  awareness 
SSS«»rporating  computing  activtties  in  classes,  and  was  more  concerned  with 

scale^Tall  facutty=5.4;  users=5.2;  non-users-5.6).  The  ™*  *™?r^ 
during  interviews  in  the  information  category  was  tor  knowing  what 

available  on  campus  for  students  to  use.  as  well  as  kinds  of  programs  being  publ ish f  by  software 
developers.  Other  concerns  were  about  acquisition  of  specific  resources^  such ijj  a  movabte 
computer  demonstration  station  with  a  projection  pad.  As  one  ^f^ff^^^J"  Concern 
demonstrate  teaching  with  computers]  if  we  don't  have  anything  [hardware  and  software].  Concern 
Sre^dKfunding  for  'resources  for  faculty  needs.  One.expenenced  instrucW 

^^SSS^^  rated  item  on  this  sub-scale  was  a  concern  about  "inability  to  manage 
all  the  in^atfon  requires  (means:  all  facu.ty=3.9;  users^.5; 

computer  laboratories  and  quantities  of  software  were  addressed.  One r  wrtoent  vi^  wn^»mea 
abouTbeing^le  to  schedute  a  classes  in  laboratories  so  that  students  and  pwtessoroo "W examm 
software  as  a  group.  Another  stated  that  teacher  candidates  should  use  computers  wrth  children  in 

actual  classrooms  rather  than  only  in  an  education  courses.  „,  „4i„a«™  n<  lacks  and 

One  interesting  result  was  the  rating  by  non-users  of  3.3  for  the  *em  "Coordination  of  tasks  and 

atteclSe^Tmeans:  aH  facu.ty=5  3;  users  6.7,  non-users=4.6).  Nearty  all  mntotfi  made  a 
statement  expressing  the  need  for  candidates  to  know  how  to  U*  "XFJtSESS* er 
emphasized  f  amiliahty;"  whereas  instructional  users  emphasized  effective  use  of  computer 
activities  and  had  more  intense  concerns  about  impact  on  students. 
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fifiUabaaligS  The  highest  rated  item  in  this  category  was  tor  wanting  "to  coordinate  my  eftorts 
with  others  to  maximize  the  innovation's  effects"  (means:  all  faculty  4.9;  users=5.  l  non-users=4  7) 
This  category  is  for  comments  about  working  wfth  other  faculty.  One  comment  indicated  we  should 
-iind  out  what  teachers  in  the  area  are  doing  with  computers."  Another  said:  The  tacutty  needs  to 

S  ?I W    *6y  f  aCt"aJ'y  66  in  courses  "  Attnoufln  Ws  category  received  the  lowest  intensity 
rating  by  non-users,  four  of  them  indicated  a  willingness  to  include  computer  activities  if  they  could  get 
some  help  in  using  computers  and  developing  assignments.  There  was  also  interest  in  knowing  what 

o  It  fJ ^~»9£  d.0lfV  PaftiCUlar'  ^  sch001  ,eacners  in  tne  area-  One  indicated  that  there 
ought  to  be  collaboration,  but  it  would  be  difficult  to  find  the  time 

RflpgysipQ.  The  highest  rated  item  on  this  sub-scale  was  about  modifying  "use  of  the  innovation 
based  on  the  experiences  of  our  students"  (means:  all  faculty=4.3:  users  5.0  non-users  3  4) 
inSSr?^.^  ^P^350*  *•  <ualitV  «  current  materials  which  some  felt  were  inadequate 
S  ™t  If       "FT ?uqUalrty  educational  so^e  applications,  and  were  not  representative 
SSJ^JJL*  1     l  *  lhere  WaS  no  clear  plan  for  inte9ratin9  «mPU»  activities  in  courses  in 
Si  1?  !  **?  aboul requirin9  computer  C0M  or  complex  of  computer  actitfty 
checklists.  Rather  than  refocus,  several  addressed  the  need  for  developing  a  focus  or  plan  There 

r^uirt?toCuseT1entS  m°m{Sl  Le"  ^  ^  »•  Macintosh,  that  students  should  be 

Implications 

»^V£?  ^  Q.is.  9en?ric  30(1  aPP'icab'e  to  many  innovations,  this  study  also  provides  information 
^InV^STl? 680,1  ■? the        sta96S  °<  witnln     context  oTeduSaT 

^K?t.ltrther  pr0V,deS  an  examp)e  of  the  aPP'ication  of  CBAN/1  theory  in  the  context  of  higher 
education  teacher  preparation  programs.  Using  Fuller's  (1969)  developmental  framework  o !  si  task 

22?  JT61  COnCerns' ,he  1acutty  38  a  whole  had  the  intense  as  well  a ^m?EiS  aSe^n 
^KSlCOnCfn^,e90r,es-  11  should  *  emphasized  that  in  many  cases  there  were Xy 

SeSSed  2  aSttmenfrCl!?881  and  86000(1  hiah  int6nsi*  ^  The  locS  was  ^ ' 
rrPv  P  5n. Instrurner,t  t0  diagnose  concerns  so  that  the  concerns  could  be  addressed  it  was 

E*SES£^        or  ,ua5? on  ,he  basis  of  ,he  "S^^SSiclSto 

m£E?t £25?  " the  number  01  respondents  in  this  study  was  small  as  well  as  being  a  non- 
random  sample  The  innovations  that  were  used  for  developing  the  scales  and  norms  Seared  to 

mSJl  K  «    •  tniMudv-t the  innovation  was  still  in  the  definition  and  design  phase 
USSST  tne^eaning  of  the  phrase  "Integrating  Computers  in  Teacher  Education Courses"  would 
^SSAt^S" wdar  'fetation  than  a  more  ctolydaRned  innovation  This maybe 
problematical  to  compare  this  innovation  with  innovations  on  the  percentile  scales  Further  studies  ot 

dSS.SJrSS^i  Wrth  8<  hlSt°^  01  '^P'ementation  would  be  beneficial.  In  such  contexts 
ofSS  W0U,d  be  in  "a<^  maintain  imp.ementa.ion 

the  riS^nTh^ofn^^"1 1actor  int1luencing  intensity  of  stage  of  concern  was  whether  or  not 

Instructional  non-users.who  may  use  computers  as  personal  tools  but  had  not  included 
"SSSS^SSS^ in  •C°UrSeS' had  ,he  *9hest  self  corKerns  None  had  S£  concerns 

COmP°nentS  With°Ul  80,1,6  ^an1?tonn^ 
pamapate  m  some  way  in  increasing  computing  abilities  of  other  faculty  and  studST  W",inflneSS  10 
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Introduction 

This  paper  has  three  main  purposes.  The  first  purpose  is  to  analyze  the 
strengths  and  weaknesses  of  three  prevailing  evaluation  models,  with  special 
attention  to  the  role  of  feedback  (overt  or  covert)  in  each  paradigm.  The  second  purpose 
is  to  present  a  framework  for  analyzing  issues  faced  by  cvaluators  of  interactive 
instructional  technologies.  Finally,  the  implications  of  applying  this  framework  to 
macro-level  evaluation  of  interactive  instructional  technologies  will  discussed. 

Questions,  Questions,  Questions 
Professionals  in  the  business  of  affecting  human  performance  cannot  escape 
evaluation.  When  making  program  decisions,  a  systematic  process  of  judging  must  be 
used.  Compound  the  pressure  of  instructional  decision-making  with  ever-changing 
interactive  technology  delivery  systems,  and  the  inherent  problems  surrounding 
evaluation  expand  into  an  exponential  migraine. 

With  these  problems  come  a  wave  of  questions.  Where  do  you  start?  Who  are 
your  real  clients?  How  do  you  decide  upon  the  major  questions  to  structure  your  study? 
How  do  you  address  hidden  agendas?  How  sufficient  are  objectives-based  evaluations 
when  working  with  innovative  programs?  How  do  you  balance  quantitative  and 
qualitative  data  collection  methodologies?  Sure,  it's  cheaper  than  hiring  outsiders- 
but  does  formative  internal  evaluation  really  work?  How  do  you  decide  the  evaluation 
is  complete?  Most  importantly,  after  you  have  conducted  this  study,  how  do  you 
feedback  your  hard-earned  evaluation  results  to  critical  audiences?  Call  these 
professionals  what  you  will-  trainers,  instructional  designers,  course  developers, 
instructional  technologists,  curriculum  planners,  teachers,  program  cvaluators-thoy 
have  not  yet  reached  consensus  about  how  to  answer  these  questions. 

Since  the  late  1960s,  authors  have  addressed  the  need  for  systematic  program 
evaluation  models.  Undoubtedly,  the  use  of  models  to  order  educational  processes 
facilitates  the  conception  of  problems  and  the  perception  of  interrelationships  within 
these  problem  contexts.  While  each  model  may  view  the  same  set  of 
interrelationships,  it  is  inevitable  that  certain  concerns  of  the  model  builders  will 
differ  due  to  personal  frames  of  reference  (Alkin,  1968).  Dillon  (1981)  extends  this 
personal  dimension  of  modelling  to  include  "prefabricated  configurations  of  concepts"  or 
"patterns  of  coherence  that  we  expect  to  find." 

The  modeling  of  social  systems  involves  another  potential  danger.  Education's 
use  of  analog  models  drawn  from  the  "hard"  sciences  is  troublesome  to  the  extent  that 
empirical  models  represent  only  "observable"  realities.  The  human  dimension  in 
instruction  requires  models  which  represent  both  observable  and  subjective  realities. 
This  dilemma  of  inherent  change  is  further  compounded  in  evaluation  when  the 
evaiuator  threatens  to  question  existing  structures,  make  covert  values  visible,  and  test 
commonly  accepted  myths  (Tucker  &  Dempsey,  1991). 

In  1971,  Daniel  Stuffiebeam  raised  several  issues  of  concern  to  cvaluators. 
These  issues  have  yet  to  be  addressed  adequately  by  prevailing  evaluation  models.  He 
notes  four  problems  in  the  application  of  experimental  designs  to  evaluation.  First, 
selection  of  experimental  designs  conflicts  with  the  principle  that  evaluation  should 
facilitate  continuous  program  improvement.  According  to  a  colleague  of  Stuffiebeam, 
"experimental  design  prevents  rather  than  promotes  changes  in  the  treatments  because 
...  treatments  cannot  be  altered  if  the  data  about  differences  between  treatments  arc  to 
be  unequivocal"  (Guba,  1969,  p.  34).  Second,  traditional  research  methodologies  arc 
useless  for  making  decisions  during  the  planning  and  implementation  of  a  project. 
Rather,  the  stress  on  controlling  operational  variables  creates  a  contrived  situation  and 
blocks  the  collection  of  natural  and  dynamic  information.  Third,  the  typical  research 
design  does  not  apply  to  the  "septic"  conditions  of  most  evaluation  contexts.  The 
evaiuator  is  not  interested  in  establishing  highly  controlled  conditions  within  which 
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of  in»i.cd  lnl«fe«~e_(J« >•»  X  "  to,  point,  internal  validity  is  gained 

instructional  technology  programs. 

r^^S:"^  heavily  on  three  ntc^ls  developed 

u  ~^/iqa7  iQft?)  and  Scriven  s  eoal-free  model  UVo/, 

em  hases  placed      ach  of  these 

tasks  Anoter Zcetf  variance  is  how  the  evaluator  relates  ^^^^ 
tasks.  An°^r  analysis  of  evaluation  models  indicates  that  large,  complex, 

SSt^liSC  £j  require  a  dl«^  moddj, 

Jo  answTr ^ti^  questions  should  be  both  quantitative  and  qualitative,  including  the 

S'St™^^  each  approach  which  have ^ 

ovor  the  ensuine  years  Rather  than  focus  on  >heir  well-publicized  strengths,  the  rest  of 
Ts  ^Horw  u8slmarize  some  inconsistencies  or  areas  which  have  not  been 
ddrSb^eah  model.  This  critique  is  motivated  by  the  needto  understand I  why 
evaluauVn  oHnteractive  instructional  technology  projects  has  been  fraught  w  th 
dlftuhies  and  !s  often  a  dissatisfying  experience  for  both  the  evaluator  and  the 
evaluated .  ^  ,n  yarious  evaluation  concerns, 

nuanTi  afivTand qualitative  Jata  sources  to  portray  a  more  holistic  reality; 
helDfu lness  toward  program  improvement;  the  evaluated  openness  to  make  values 
SJffjTuSState  the  planning  of  probable  futures  for  a  program  .The  rest  of  this 
secTon  analyzes  the  strengths  and  weaknesses  of  eacb .  model »  more  depth. 

Insert  Figure  1  about  here 


1  Instructional  Objectives  Approaches 

Ws  model  has  several  merits.  It  provides  valid,  reliable,  and  objex :  we 
data  for  an  evaluation.  It  allows  the  evaluator  to  indicate  attained  and  unatta.ned 
ob  ectTves  On  the  other  hand,  strict  application  of  the  Tyler  Model  creates 

Uascertains  student  outcomes  but  ignores  the  contexts  and  processes  that 
fead  toTes^ 'SSSSim.  The  statement  of  objectives  in  behavioral  terms  is  a  long 
and  ortenledious  procedure  which  limits  sources  of  evidence  to  purely  quantifiable 

1  ^sence  it  is  too  easy  to  avoid  questions  about  the  worth  of  the  objectives 
«lvrSs  is ■  c^3y  true  if  the  evaluation  is  conducted  after  the  objectives 
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are  formulated.  When  this  happens,  the  evaluation  is  often  limited  to  making 
superficial  revisions  of  performance  objectives. 

In  response  to  these  difficulties,  more  flexible  "neo-Tylerian"  models  have 
emerged  (Taba  &  Sawin,  1962;  Cronbach,  1963;  AAAS  Commission  on  Science  Education, 
1965).  Tyler's  data  sources  have  been  expanded  to  include  observations  on  teaching 
method,  patterns  of  classroom  interaction,  physical  facilities,  and  student  motivation. 
MNeo-TyierianM  ideas  have  contributed  to  the  field's  shift  from  a  terminal  focus  to  one 
which  synthesizes  both  process  and  product  elements.  The  emphasis  of  evaluation  is  on 
facilitating  instructional  improvement.  Two  major  limitations  persist.  One,  the 
evaluatbrs  do  not  incorporate  client  feedback  into  the  proposed  evaluation.  Second, 
futures  planning  is  neglected. 

Z  Decision  Making  Approaches 

A  second  group  of  evaluation  theorists,  notably  Provus  and  Stufflebeam,  believe 
the  evaluator's  task  is  one  of  delineating  information  to  be  collected,  planning  a  data 
collection  methodology,  and  helping  decision  makers  use  the  information  collected.  It  is 
the  responsibility  of  the  decision  maker,  not  the  evaluator,  to  make  judgments  from 
this  information.  Four  questions  arc  basic  to  this  approach: 

•  What  should  the  objectives  be? 

•  How  should  the  program  be  designed  to  meet  these  objectives? 

•  Is  the  program  design  being  carried  out  effectively  and  efficiently? 

•  Was  the  program  worth  what  was  invested  in  it  considering  the  products 
achieved?  (Reasby,  1973,  p.  23). 

While  advancing  the  field  by  specifically  addressing  standards,  Provus  (1971) 
identified  several  weaknesses  in  his  model.  He  concludes  that  the  major  weakness 
with  the  discrepancy  model  seems  to  stem  from  failure  to  understand  the  limits  on 
institutional  behavior  set  by  field  conditions,  the  intricacies  of  the  decision-making 
process,  and  the  origin  and  use  of  criteria  in  that  process.  Context  evaluation  is  not 
addressed  and  the  exhaustive  use  of  behavioral  standards  may  limit  the  creative 
adaptive  responsiveness  of  a  program  staff.  Provus's  approach  does  not  clearly 
identify  the  type  and  ultimate  use  of  information  to  be  collected  at  each  stage.  The 
model  could  also  be  faulted  for  being  incapable  of  evaluating  rapid,  large-scale  changes 
characteristic  of  early  instructional  technology  projects.  Decision-makers  arc  not 
always  rational,  yet  the  model  assumes  such  behavior.  Finally,  the  Discrepancy 
Model  does  not  address  how  evaluators  are  recruited,  trained,  and  maintained  in  the 
system. 

The  Phi  Delta  Kappan  Committee  on  Evaluation,  chaired  by  Daniel 
Stufflebeam  (1971 ),  has  perhaps  exerted  more  influence  than  any  group  in  attempting  to 
lead  the  educational  profession  away  from  excessive  reliance  on  classical  research 
models.  Instead,  an  evaluation  model  is  offered  which  assists  decision  makers  in 
pinpointing  their  values  so  that  they  can  best  be  served  by  the  decisions  made  This 
model,  known  as  C1PP,  specifies  four  types  of  evaluation:  Context,  Input,  Process  and 
Product  (Stufflebeam,  1983).  "  " 

Though  mechanically  total,  the  C1PP  model  excludes  overt  values  in  its 
schema.  According  to  Stufflebeam,  the  evaluator's  role  is  one  of  collection, 
organization,  and  analysis  of  "relevant"  data  for  "relevant"  decision  makers  Overt 
judgment  of  the  intrinsic  worth  of  the  program's  oojectives  is  not  considered  by  the 
evaluator  or  the  clients.  The  entire  question  of  values  is  kept  tacitly  in  the  decision- 
makers' domain.  Another  limitation  of  the  C1PP  model  is  its  inability  to  answer  two 
basic  questions.  How  do  evaluators  and/or  clients  know  which  standards  are  operant? 
Further,  what  processes  are  necessary  to  enable  the  decision  maker  to  apply  value 
criteria?  rv  J 
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VoZ^X"*^**  aPP-CheS  t0  Pr°8ram  eVa,Uati0H  arCa 
■a  J  ^na^T  sTriven's  Goal-Free  Model  and  Stake's  responsive  schema. 

COm,dCr^?  (196?  m TSm  Goal-Free  model  is  in  definite  contrast  to  those 
[he  anxiety  of  those  being  evaluated.  The  consequence  of  this  type _  of  di stortea 

-r^sTs^ 

oval»,»^n  ^,0  or  .he  null's  s,rc„g.hS,  several  q"f!<™  ™™> ^  f 
1«2>  is  to.  a^valua  Jrnus.  do  Ms  best  to  reflect  the  nature  of  the  programand  no. 

^vt°'S"  IC  fhfevaluator  should  no,  take  an  »bso,„.is, 

£s«ton  regarding  the  program's  goals  because  this  is  likely  to  make  Cents  kss 
wining  »«J^^aSr»*li  in  attempting  to  describe  and  urdgc  the 

^IZ'c  is^menTof  evaluation  procedures  in  the  descriptive  matrix  »  unclear  as  .t 

«S=g'^^r,^»^-'" 

cxSnce  ?ur,fermoreghedocs  no.  provide  for  sys.ema.ic  feedback  .np»g»m 
at  evaluation  from  the  terminus. 

Furthermore  evahiation  is  only  as  good  as  the  environment  that  the  evaluator  is  able 

among  all  significant  players.  Approaching  this  cooperate 
Sa  c  can  facilitated  by  PolanyTs  (1962, 1966,  1975, 1978)  constructs  of  collecting 
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multiple  perceptions  in  order  to  approach  a  shared  reality.  He  also  advocates  trying  to 
make  tacit  perceptions  overt  rather  than  being  unconsciously  controlled  by  these 
perceptions,  and  including  extended  dwelling  in  an  evaluation  to  experience  more  than 
the  surface  and  first  impression  phenomena.  In  essence,  out  of  these  diverse  perceptions 
we  can  identify  critical  polemics  as  signposts  which  expand  but  do  not  fix  our 
capability  to  describe  and  judge.  That  is,  we  can  use  these  polemics  to  avoid  a  closed 
view  of  evaluation  and  open  our  perspective  to  the  variety  of  factors  which  can 
influence  evaluated  and  client's  judgments.  Figure  2  summarizes  some  of  these  major 
polemics  and  their  continua.  ' 

Insert  Figure  2  about  here 


The  Need  for  a  Different  Model  to  Evaluate  Instructional  Technologies 
As  summarized  in  Figure  2,  the  three  prevailing  approaches  possess 
similarities  and  differences.  Special  attention  is  directed  to  the  presence  or  absence  of 
several  criteria:  (1)  using  a  systematic  methodology  which  combines  quantitative  and 
qualitative  data  sources  to  portray  a  more  holistic  reality;  (2)  helping  toward  program 
improvement;  (3)  providing  evaluator's  feedback  to  make  values  overt-  and  (4) 
facilitating  program  planning  regarding  probable  futures.  To  varying  degrees  the 
instructional  objectives  approach,  the  decision-making  approach,  and  the  values-based 
equator?  methods  for  systematizing  the  feedback  of  both  the  evaluated  and 

Given  the  aforementioned  variance  in  evaluation  models  as  well  as  diversity  in 
clientele  and  program  purposes,  it  is  apparent  that  judgments  about  program  worth  can 
vary  accordingly.  It  is  the  contention  of  this  paper  that  the  perceptions  of  both  the 
evaluated  and  the  evaluators  need  to  be  made  as  overt  as  possible.  This  coenitive 

1  ,h(e  "?  °f  [nformed  jUdgmcnt  makh«  and  Prog-™  improvement. 

Support  for  this  position  has  been  persuasively  advanced  by  Weckworth: 

"First,  there  is  no  one  way  to  do  evaluation;  second,  there  is  no  generic 

logical  structure  which  will  assure  a  unique  'right'  method  of  choice.  Third 

evaluation  ultimately  becomes  judgment  and  will  remain  so,  so  long  as  there  is 

no  ultimate  criterion  for  monotonic  ordering  of  priorities;  and  fourth,  the 

critical  element  in  evaluation  is  simply:  who  has  the  right,  i.e.,  the  power,  the 

influence,  or  the  authority  to  decide."  (1969,  p.  48). 

ft,  t   t  U  iV^r?05?  °f  thiS  SCCti0n  10  ofrcr  an  intcractive  evaluation  model  (Kunkel 
&  Tucker,  1983;  Tucker  J  pempsey,  1991)  which  addresses  some  of  the  weaknesses  of 
the  aforementioned  models  and  strives  for  credible  synthesis  of  their  strengths.  This  is 
especially  difficult  to  achieve  in  contexts  reflecting  diverse  client  and  audfence  value? 
such  as  instructional  technology  programs.  A  macro-level  evaluation  model  is 
therefore  perceived  as  a  body  of  interrelated  criteria.  These  criteria  help  determine 
ma,or  questions,  sources  of  evidence,  and  standards  for  judging  the  findings  and  making 
recommendations.  Finally,  it  is  informed  by  the  research  base  of  recent  developments  in 
scSs  2  PSyCh0,°8V-  F'gUrC  3  su™arizes  the  cognitive  complexity  of  this  process  as  a 

•  contextual^  relevant  exchanges  between  information  inputs  and  feedback 
messages  between  the  evaluation  and  clients; 

«  perceptual  representations  of  this  data;  and 

•  resultant  actions,  responses  and/or  judgments  regarding  the  data. 

Insert  Figure  3  about  here 
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Bv  making  overt  the  vulnerable  judgments  of  both  evaluators  and  those  being 
negotiated.  Through  cycles  of  (^cVte\wwtoeeja  embodied  in  the 

judgments  are  made.  ...     Drooosed  ceriain  value  criteria  are  non-negotiable 

oriented  and  futuristic. 

adeCy  of  thfprogram.1^ It  encourages  developing,  revising,  and  refimng task 

SteMhiT^tential  bias,  an  evaluaHon  model  (see  Bgure  3)  .s  proposed  that. 

•is  interactive  and  negotiated  between  the  clients,  evaluators  and 

Joa'sXThe  perceptions  and  values  of  many  participants  to  be  capture  a 

hoiistie  rather  than  positivistic  reality; 
.involves  significant  players  throughout  all  four  phases,  not  ,ust  at  the 

.X"Syd-"»^k  »  —  i*'"a"OM'  a"d  mo"va,io"a' 


functions. 
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Phase  Is  Generating  Questions  and  Setting  Standards 

As  depicted  in  Figure  3,  Phase  I  has  two  levels  of  feedback  loops.  These  loops 
consist  of  perceptual  exchanges  between  the  clients,  evaluators  and  relevant  audiences 
These  exchanges  involve  the  perception  of  the  task  demand  of  the  client  and  major 
questions  synthesized  by  the  evaluator  to  focus  the  evaluation. 

Consider  the  first  feedback  loop.  Here,  the  client  introduces  the  task  demand  to 
the  evaluator.  The  client  s  presentation  can  be  influenced  by  both  overt  and  covert 
factors,  how  stable  the  client's  organizational  environment  appears  to  be,  prior  history 
of  experiences  with  evaluation,  and  general  belief  structures.  It  becomes  readily 
apparent  that  evaluation  serves  many  functions  besides  error  reduction.  For  example  it 
can  provide  *  ridence  of  the  client's  competence,  signal  impending  action,  and  can 
protect  the  client  politically. 

Next,  the  evaluator  perceives  the  task.  This  perceptual  representation  is 
shaped  by  a  variety  of  factors.  These  factors  include  the  evaluator's  prior  work 
experiences,  his  ability  to  decode  semantic  and  semiotic  messages,  the  perceived 
priority  or  strength  of  demand  of  the  task,  recognition  of  political  forces,  confidence, 
and  stakeholder  boundaries. 

The  first  output  of  the  evaluator  is  an  attempt  to  identify  the  program's 
operational  questions  and  standards  expressed  by  the  client.  This  could  be  done  at  any 
phase  of  He  instructional  process  (e.g.,  analysis,  design,  development,  implementation 
or  dissemination).  This  completes  the  first  cycle  of  information  inputs. 

If  an  evaluator  uses  only  the  first  feedback  loop  to  develop  his  questions  and  set 
standards,  the  risk  of  perceptual  misunderstandings  between  the  evaluator  and  client 
are  still  present.  Recognizing  this  risk,  experienced  evaluators  often  confirm  their 
identification  of  the  major  questions  by  seeking  feedback  from  the  client.  Clients  are 
asked  how  relevant  these  questions  are  to  their  interests  and  needs  (versus  a  more 
research-oriented  perspective  which  would  give  the  evaluator  sole  authority  to 
determining  what  is  valuable  and  relevant).  Factors  which  can  influence  the  client's 
feedback  message  include:  the  client's  expectations  about  short  and  long  term  program 
outcomes,  the  personal  significance  of  the  questions,  penalty  costs  of  whether  or  not  to 
gather  the  information,  the  client's  perception  of  the  evaluator  s  degree  of  expertise 
and  rapport  established  between  the  client  and  evaluator.  '  ' 

When  feedback  is  readily  available,  the  risk  of  serious  initial  judgment  errors 
is  not  as  high  because  feedback  provides  information  upon  which  to  base  corrective 
adjustments.  Consider  Hogarth's  (1981)  likening  this  process  to  aiming  at  a  target  The 
evaluator  may  engage  in  a  series  of  exchanges  with  the  client.  The  intent  of  these 
exchanges  is  to  focus  the  evaluator  s  perception  of  the  task  or  "target."  Then  the 
evaluator  forms  a  mental  representation  of  the  perception.  For  example,  in  the  first 
phase  of  evaluation,  this  representation  serves  to  eliminate,  substitute,  or  strengthen 
the  evaluator's  perceptions  about  the  questions  being  asked.  For  example,  perceptions 
of  any  hidden  agendas  emerging  from  the  first  cycle  of  information  inputs  are 
considered  Given  the  prior  experience  of  the  evaluator,  the  task  may  be  encoded  as 
objectives-based,  management-based,  or  values-based,  or  a  composite  of  all  three 

Finally,  this  cycle  ends  with  the  evaluator  negotiating  an  evaluation  plan. 
This  plan  or  paradigm  serves  as  a  blueprint  to  guide  the  ensuing  evaluation.  Three 
components  make  up  this  paradigm:  questions,  evidence,  and  standards.  Questions  to 
guide  the  study  are  framed,  potential  sources  of  evidence  clarified,  and  standards  or 
qualities  by  which  to  judge  the  findings  are  made  overt.  Negotiation  of  the  paradigm 
1    ,!  I0"",     sys,ematic  Participation  of  all  relevant  stakeholders.  This  negotiation 
should  be  done  during  the  first  phase  of  the  evaluation,  thereby  reducing  some  of  the 
perceptual  errors  inherent  in  models  such  as  Scriven's  goal-free  approach.  It  has  been 
the  experience  of  the  author  that  negotiation  serves  as  the  first  step  in  co-opting  even 
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the  most  negative  stakeholders.  This  is  accomplished  by  being  open  to  their  critical 
questions  and  emerging  perceptions. 

Implications  of  Phase  1 

Helping  stakeholders  generate  questions  is  affected  by  one's  capacity  and 
motivation  to  generate  alternative  hypotheses  for  a  given  question.  (For  more  thorough 
discussions  of  hypothesis  generation  refer  to  the  work  of  Higgins  &  King,  1981;  Eco  and 
Scbcok,  1983;  Fisher,  Gettys,  Maming,  Mehle,  &  Boca,  1983;  Mayseless  &  Kruglanski, 
1987.)  For  example,  in  situations  where  time  pressures  are  great,  a  client  or  evaluator 
may  be  tempted  to  focus  solely  on  addressing  pre-defined  objectives.  In  settings  where 
formative  evaluation  provides  useful  feedback  for  product  development  and  revision, 
its  design  is  often  constrained  by  available  resources  such  as  time,  money,  personnel,  and 
facilities. 

Extending  the  example  of  formative  evaluation  to  interactive  instructional 
technology  further,  it  appears  that  two  major  questions  are  posed.  One  deals  with  the 
content  and  technical  quality  of  the  product  and  the  other  deals  with  its  learnability 
within  authentic  contexts.  More  specific  questions  generated  by  these  two  foci  could 
deal  with  product  initiation,  design,  content  development,  media  development, 
production,  testing  and  maintenance  (Foshay,  1984).  Geis  (1987)  suggests  that  feedback 
is  needed  regarding  the  content  and  sequencing  of  instructional  events  as  well  as  the 
nature  of  -arner  control  of  the  content  or  message  as  sent.  Consider  the  case  of 
instructional  hypermedia.  Table  1  summarizes  just  some  of  the  questions  that  could 
emerge  when  one  is  open  to  more  holistic  views  of  evaluation,  hopefully  more 
accurately  reflecting  the  medium's  true  nature. 

Insert  Table  1  about  here 


Besides  these  two  major  questions,  each  stakeholder  appears  to  have 
idiosyncratic  questions  of  interest.  Rather  than  reaching  consensus  on  the  questions,  an 
effort  is  made  to  solicit  the  complete  realm  of  questions  and  then  synthesize  them  into 
broad  questions  which  allow  multiple  perspectives.  Fifteen  years  of  evaluation  field 
narratives  suggest  this  is  a  viable  strategy  and  management  research  seems  to  support 
this  as  well.  The  research  of  Walsh  and  associates  (1988)  argues  that  increased 
coverage  and  decreased  consensus  are  important  early  in  the  decision  making  process  to 
help  define  the  problem  or  task.  Once  the  group  comes  to  understand  the  information 
environment,  however,  a  consensus  around  a  narrower  view  (i.e.,  evaluation  paradigm 
negotiated  during  feedback  loop  2)  is  beneficial.  The  ability  to  read  a  decision 
environment  and  capture  the  major  evaluation  questions  and  the  belief  structures  behind 
these  questions  is  the  essence  of  shared  veridicality. 

Questions  are  also  influenced  by  an  individual's  capacity  to  see  the  values  or 
standards  that  are  behind  the  questions.  Why  was  the  question  asked  in  the  first 
place?  Standards  describe  the  results  that  are  desired  or  should  be  achieved  upon 
satisfaction  of  a  task.  To  facilitate  performance,  researchers  contend  that  standards 
need  to  be  specific  as  well  as  identify  qualities  and  quantities  of  output  expected 
(Baird,  1986).  Specific  standards  serve  as  motivational  devices  for  individual 
performance  (Locke  ct  ah,  1981;  Taylor,  1983)  and  can  anchor  future  decisions  of  the 
client  (Huber  &  Neale,  1986).  This  phenomenon  may  explain  the  evolution  of  "building 
block"  evaluations  which  focus  on  lower  level,  fragmented  tasks  which  are  more  easily 
identified  and  documented. 

Instructional  designers  lack  Jonsistent  formative  evaluation  guidelines 
regarding  products  that  aim  at  interactive,  integrative  skill  development.  It  is 
difficult  (but  not  impossible)  to  evaluate  situations  which  allow  learners  to  practice 
multiple  skills.  Compounding  this  complexity  is  the  presence  of  unpredictable 
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navigational  paths  and  a  need  to  assess  variable  performance  standards.  In  fact, 
interactive  instructional  design  seems  fraught  with  many  aversivc  prior  learning 
experiences.  For  starters,  many  managers  expect  cost  overruns  and  schedule  slippage. 
Another  fear  involves  the  losses  attributed  to  the  new  product's  possible  failure 
spreading  to  established  products.  Finally,  while  many  designers  recognize  that 
prevailing  linear  and  iterative  strategies  result  in  "piecemeal"  products,  they  lack 
alternative  design  methods  peculiar  to  this  technology. 

Program  and  product  performance  is  compared  to  a  norm  or  standard.  Usually, 
the  norm  is  the  maximum  achieved  by  an  optimal  allocation  policy.  Invariably, 
performance  is  found  wanting  (Busemcyer,  Swenson,  &  Lazartc,  1986).  Optimal  policies 
cannot  be  specified  without  perfect  knowledge  of  the  goal  state.  For  many  real 
decisions,  however,  only  imperfect  and  vague  information  about  the  objective  and 
solution  is  available.  Even  if  specifications  exist,  management  may  not  have  made  it 
clear  whether  the  intent  is  to  meet  specifications  or  exceed  them.  Hence,  'here  is  a 
perceived  need  for  evaluation.  This  seems  to  be  particularly  true  of  interactive 
technologies  such  as  instructional  hypermedia  where  the  operant  criteria  have  yet  to 
be  defined. 

And  what  happens  when  the  product's  instructional  and  business  goals  are  not 
clearly  communicated  to  the  technicians  and  instructional  designers?  What  often 
results  is  loyalty  to  functional  standards  versus  the  total  plan  because  immediate 
functional  rewards  upstage  measures  of  organizational  performance.  For  example,  in 
university  environments  where  product  development  is  almost  exclusively  driven  by 
external  funding,  the  product  can  have  high  priority  within  the  grant  but  low  priority 
throughout  the  rest  of  the  institution. 

Standards  are  often  very  difficult  to  calculate,  and  it  seems  unreasonable  to 
expect  that  the  typical  client  knows  the  optimal  solution  a  priori.  For  example, 
developers'  #nd  users'  judgment  of  product  quality  extends  beyond  hardware  and 
software.  Quality  involves  technical,  educational,  and  financial  attributes; 
installation  issues;  low  maintenance;  high  reliability;  compatibility  with  other 
equipment  already  in  place;  ease  of  upgrading;  and  multi-platform  access.  Those  arc 
standards  that  are  typically  not  incorporated  into  questions  asked  by  developers. 
However,  it  is  reasonable  to  expect  that  program  managers  and  designers  can  improve 
their  standards  and  rules  for  optimal  policies  iftraining  incorporates  informative 
feedback  (Busemeyer,  Swenson  &  Lazartc,  1986).  And  the  corollary  is  that  while  this 
rational  explanation  may  be  possible,  the  cvaluators  must  be  prepared  for  instances 
where  the  client  intends  the  evaluation  to  serve  functions  other  than  an  error  reducing 
role.  For  example,  clients  may  want  the  evaluation  to  give  evidence  of  a  key  player  s 
competence,  signal  impending  action,  or  "cover  their  tracks." 

Expectations  about  future  outcomes  strongly  affect  one  s  decisions  ( Abclson  & 
Levi,  1985;  Feather,  1982).  Developers  often  devote  much  time  examining  the 
implications  of  alternative  designs  and  the  adequacy  of  the  likely  reward  in  view  of 
the  risks  incurred.  Because  of  the  importance  of  such  expectations,  much  research  has 
been  devoted  to  understanding  how  individuals  estimate  probabilities  of  future  events. 
Evidence  has  accumulated  that  individuals  use  a  general  cognitive  strategy,  the 
"availability  heuristic"  (Tversky  6.  Kahncman,  1973).  In  using  this  heuristic, 
individuals  estimate  the  probability  of  events  by  the  ease  with  which  they  can  recall 
or  cognitively  construct  relevant  instances.  Specifically,  questions  can  serve  as  a 
valuable  heuristic  or  cognitively  simplifying  strategy  for  participants  in  the 
evaluation  process. 

Not  surprisingly,  our  field  experience  indicates  thai  decision  makers  use  only  a 
subset  of  available  information  in  evaluating  a  program.  Rather,  managers  seem  to 
rely  on  heuristics  to  simplify  complex  environments.  While  serving  to  avoid 
information  overload,  inappropriately  applied  heuristics  can  provide  a  significant 
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source  of  bias  and  error  for  decision  outcomes.  For  instance,  V^^^f^  wC 
overwefch  certainty  in  their  decisions  (Kahneman  &  Tversky  1979).  As  evaluator ,*c 
ZuTd  amidpate  mis  heuristic  when  clients  generate  evaluahon  queshons-^nd  strwe 
fnr  nuPs^nsThat  capture  the  task's  complexity  rather  than  satisfy  predetermined 
ZSSSSS  SS  systematic  use  of  shared  cognise  feedback  and  negouauon  to 
^leTse      Questions  is  seen  as  an  effective  strategy  to  mmmuz*  ttus  bias^ 
cleanse  me  q  tion<can  lead  to  a  perceptual  bias  which  may 

ootentiaU Tbe  dVdted  by  either  task  or  context  characteristics.  Kahneman  and 

Bazerman,  1985). 

f  hacf  ?r  n^rripti^n  "f      St""rps  an(i  Analysis  of  Alternatives 
S^foXultimate  choices  made  hinges  upon  how  well  the  <*ents  and 
eva.ua  JsXbJfo  attend  to,  encode,  store,  retrieve,  and  f™ 

on  *  w^rn -ten  influence  information  receptivity, 

Cr^ibi,iSfolStely,  this  process  is  not  easy.  A  piece  of  information  must  pass 
through  many  filters  before  it  is  recorded  or  retrieved.  Dec ision  ^ers  have  a 
Propensity  to  use  only  a  subset  of  available  information  when  judging  a  program  s 
ST  This  irregular  (and  often  irrational)  process  may  be  more  systematic  if  feedback 
,rpS  are  uid.  This  loop  or  cycle  consists  of:  (1)  information  inputs;  (2)  perceptual 
remcsentations;  and  (3)  outputs  regarding  the  evidence  gathered  to  answer  the 
;SCsd  in  phase  one.  Two  feedback  loops  appear  to  operate  dunng  this  second 

PhaS°"  in  the  first  feedback  loop  in  this  phase,  the  evaluator  collects  data.  This 
process  is  ffec  5  by;   tne  stability  of  the  organizational  environment  the  length  o 
the  t"^e  the  evaluation  has  committed  to  dwelling  in  the  system;  and  the  nature  of  the 
bahnce  between  qualitative  and  quantitative  evidence.  Acquiring  mformation  prior  to 
alecTsion  haslwo  major  risks.  First,  a  tenuous  evaluator  could  overacqu.re  information 
and^rexcessivecosts.  Second,  an  overconfident  evaluator  might  underacquire 
information  and  incur  excessive  risks  of  dedsional  error.  In  any  even  ,  the  costs  of 
aSg additional  information  may  be  immediate  and  easy  ^^^^ 
Sncms  of  doing  so  can  be  unclear  and  often  long  delayed  (Connolly  &  Thorn,  1987). 

Next  the  evaluator  analyzes  the  obtained  data.  This  representation  is 
affected  W    myriad  of  factors  such  as:   the  evaluated  information  oad  capacity;  his 
auaience  analysis;  the  amount  of  antidpated  versus  unanticipated  mformation 
revealed;  audience  and  client  analysis;  and  whether  a  compensatory  or 
noncompensatory  data  synthesis  model  is  being  used  (Einhom  &  Hogarth,  1981,  Billings 
&  M?rcT  1983)  Noncompensatory  models  minimize  the  questions,  criticisms  and 
LeSdata  Compensatory  models  allow  the  mpresentation  of  cognitively  complex 
1nH  coohisticatod  strategies  for  information  integration. 

and  .opta*^^  g^  ^  ^  ^  ,  data 

findings  to  the  client.  This  reporting  procedure  must  consider  many  factory Son* 
crucial  variables  include:  the  amount  of  feedback  processing  t.me  ava.lable  to  the 
client-  the  client's  relative  emphasis  on  program  documentation  versus  program 
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improvement;  and  whether  the  evidence  can  be  shared  with  the  clientele  all  at  one 
time  or  presented  over  a  period  of  time. 

Perceptual  errors  can  result  if  the  evaluator  does  not  receive  a  message  from  the 
client  regarding  the  accuracy  of  this  initial  data  report.  By  scheduling  a  regular 
exchanges,  the  client  s  initial  certitude  about  the  original  evaluation  plan  (as  well  as 
hidden  agendas)  can  be  ascertained.  In  the  process,  a  sort  of  error  checking  occurs  about 
the  perceived  accuracy  of  the  data  thus  far  collected  as  well  as  an  early  measuring  of 
the  e valuator's  credibility.  This  is  also  the  time  to  assess  the  client's  tunnel  vision  and 
blind  spots  and  selective  perception. 

The  evaluator  cognitively  represents  the  client's  messages  in  light  of 
commitments  (sometimes  complementary  and  sometimes  competing)  between  the  client 
and  evaluator.  This  representation  takes  the  form  of  either  improving  or  simply 
documenting  the  program.  A  choice  point  also  involves  the  balance  between 
qualitative  and  quantitative  data.  Obviously,  this  takes  skill  in  balancing  positive 
and  negative  information.  This  perceptual  recycling  of  data  perceptions  is  necessary  in 
light  of  research  showing  human  information  acquisition  is  often  weakly  guided  by 
normative,  cost-minimizing  considerations.  This  occurs  even  when  serious  efforts  are 
made  to  simplify  the  task,  provide  high  motivation  and  focus  attention  on  balancing 
information  costs  and  benefits.  Clients  consistently  underpurchase  "good"  information 
and  "bad"  information  is  consistently  overpurchased.  One  part  of  the  explanation  is 
client  difficulty  in  discerning  the  validity  of  different  information  sources,  even  after 
repeated  exposure.  Another  possible  explanation  is  the  certainty  of  the  cost  involved 
in  acquiring  information  as  against  the  uncertainty  of  its  benefits  in  reducing  decision 
errors.  A  risk  adverse  client  will  tend  to  overpurchase,  a  risk  seeker  to  underpurchase. 

Finally,  the  evaluator  edits  the  initial  data  analysis  and  reports  the  revisions 
along  with  the  implications  of  preliminary  judgments  given  the  evidence.  Once  again 
influencing  variables  include  the  amount  of  time  available  to  the  evaluation  for 
information  feedback  to  the  client,  the  kind  of  heuristics  which  accompany  the  data 
the  way  information  is  "framed ",  and  the  form  of  information  display. 

Implications  of  Phase  2 

Let's  continue  the  example  of  formative  evaluation  of  interactive  technology 
products  and/or  projects.  The  actual  process  of  data  collection  can  generate  as  many 
questions  as  it  answers.  Just  consider  the  following  issues: 

•  How  continuously  should  data  collection  be  conducted:  during  rapid 
prototyping,  draft  form  testing,  final  testing,  or  during  each  stage  of  the 
developmental  process? 

•  What  qualitative  and  quantitative  methods  should  be  used:  self-evaluation, 
expert  review  or  student  review  (Montague  et.  al.,  19S3);  try-out  and  revision ' 
testing  sessions  (Geis,  Burt,  &  Weston  1985);  draft  and  revise  and  field-test 
(Dick  &  Carey,  1985);  one-to-one  testing;  group  testing;  extended  testing 
(Stolovitch,  1982;  Komoski  &  Woodward,  1985);  peer  debriefing; 
triangulation;  negative  case  analysis;  and/or  audit  checks? 

•  Who  should  be  included  as  data  sources:  learners  (expert  versus  novice), 
developers,  subject  area  experts,  teachers  and  trainers,  native  informants 
audience  specialists,  gatekeepers,  sponsors,  former  learners,  editors? 

•  How  many  people  should  be  involved-what  is  an  "adequate''  size? 

•  What  are  desirable  characteristics  of  the  data  sources:  representative  versus 
select  (e.g.,  highly  verbal  or  high  aptitude),  active  versus  passive,  internal 
versus  external  to  project,  continuous  versus  one-time  involvement? 

•  Where  should  data  be  collected:  in-house  versus  field,  using  simplified 
versus  progressively  more  complex  and  less  familiar  systems? 
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.  When  should  data  collection  stop?  For  example,  should  evaluation  continue 
until  redundancy  of  learner  comments  occurs?  When  .s  the  cntenon 
performance  level  reached? 

The  client  may  accept  the  evident*  "as  is",  request  additional  evident  or  even 
,0*  ,„  & 1Z  eva.LonPFo, example,  the use  C "^/^t 

Star.  M»  s^es.  ^^^^tTJ^SS  SET 

.  only  weakly  responsive  to  various  normative  considerations  such  as 

information  quality  and  cost  (Pitz,  1968;  Wendt,  1969); 
.  suSntially  responsive  to  normatively  irrelevant  actors  such  as  total 

information  available  (Levine,  Samet  &  Brahlek  1975) 
.  slow  to  show  learning  in  repeated  decision  stations  (Lanzeta  &  Kanaret, 

1962;  Wallsten,  1968;  Busemeyer,  1985);  and 

information  (Pitzet.  al.,  1969;  Hershman  &  Levine,  1970)  though  risk  seekers 
show  a  tendency  to  underpurchase  information. 

A  two  tiered  approach  to  data  sharing  is  important  for  ^th  cognitive  and 
^.nt^ 

Additionally  the  form  of  information  display  can  encourage  or  discourage  certain 
memX  Citive  processing, given  the P^>°'h«~"f 
which  minimize  their  effort.  For  example,  organizing  data  xm 
Quantitative  information  easier  to  use,  increasing  its  impact  upon  choices  (Russo,  1977). 

SaHy  X  data  is  described  in  words  i^™-*?J 'ESSSS-WM 
abandon  strategies  which  use  mental  arithmetic  (Huber,  1980).  As  Sloyic  uw*,i! f»J 
Mtattd J* more  effort  required,  the  more  likely  it  is  that  clients  w.11 ignore  or 
SSUaH,n.  Cognitive  strain  may  such  as 

ln6  So  theluategy  to  suit  the  display  rather  than  transform  the  data  to  fit  the 
tratetr  When  evadors  recognize  this  process  and  the  impact  of  can 
£S  evaluation  data  displays****  passively  encourage  better  information 

and  hyJSSS  themselves  or  other  people  or  events),  people  predominan  ly 
heTr  pre-existing  notions.  This  implies  a  pervasive  inscnsitivity  to 
d^Sg^vidence,  and  (presumably)  a  reluctance  to  generate  alternatives.  The 
func  tions  for  how  evaluatoVs  can  share  information  with  clients  are  profound. 


1059 


Evaluating  ITT 
14 

Phaser  Making  Ttirigmonta 

Many  evaluators  neglect  to  address  their  role  in  the  act  of  judging,  preferring  to 
perceive  themselves  as  untainted  by  values  issues.  This  is  counterintuitive.  Judgment 
pervades  the  total  process,  from  the  selection  of  questions  and  data  sources  to  the 
ultimate  decisions  reached  and  choices  made.  Phase  3  presents  a  series  of 
informational  inputs,  perceptual  representations,  and  actions  in  order  to  generate 
judgments  about  program  worth. 

After  receiving  the  evaluator's  final  descriptions,  data  analyses,  and  tentative 
alternatives,  the  client  reacts  in  the  form  of  another  feedback  message.  The  nature  of 
this  feedback  will  be  tempered  by  the  client's  perceptions  of  certitude  about  the 
findings  and  expected  gains  versus  losses.  Many  researchers  arc  pessimistic  about 
managers'  abilities  to  interpret  information  from  complex  environments  accurately. 
Managers  are  thought  to  suffer  from  selective  perception  (Dearborn  &  Simon  1958) 
strategic  myopia  (Lorsch,  1985),  tunnel  vision  (Mason  &  Mitroff,  1981),  and  blind  spots 
(Murray,  1978).  ^ 

Given  the  client's  reactions,  the  evaluator  itemizes  and  begins  to  edit  the  pool 
of  possible  alternatives  in  preparation  for  generating  judgments.  Variables  that  seem 
to  influence  this  editing  process  include  the  evaluator's  capacity  to  visualize  a 
continuum  of  concrete  and  abstract  instances,  the  duration  of  events,  and  the  client's 
self-efficacy  status. 

Editing  results  in  the  evaluator  rating  or  valuing  alternatives  against  the 
standards  specified  in  Phase  One.  Besides  the  amount  of  processing  time  available  for 
the  evaluation,  what  is  at  stake  is  the  quality  of  the  alternatives  envisioned  as  well 
as  the  heuristics  used  for  conveying  these  alternatives  to  the  client  (for  example, 
availability,  adjustment,  anchoring,  and  representativeness). 

It  appears  that  some  decision  strategies  are  used  more  than  others.  The  issue  is 
frequency  of  use  rather  than  effectiveness  of  stategy.  Recent  research  suggests 
expectation  models  over-intellectualize  the  cognitive  processes  people  use  when 
choosing  alternative  actions  (Schwab,  Olian-Gottlieb,  &  Hcncman,  1979;  Fischhoff 
Goitein,  &  Shapira,  1983;  Mitchell  &  Beach,  1990).  For  example,  formal  analytic  ' 
strategies  are  seldom  used  even  by  people  who  know  about  them.  Isenberg  (1984)  adds 
that  even  when  they  use  these  logical  strategies  they  seldom  accept  the  results  that 
run  counter  to  their  intuition.  In  certain  cases,  intuition  turns  out  to  be  more  accurate 
than  analytic  strategies  (Hammond,  Hamm,  Grassia,  &  Pearson,  1987).  Mitchell  and 
Beach  (1990,  p.3)  capture  this  concept  well  with  the  following  statement:  "Formal 
analytic  strategies  require  a  great  deal  of  concentration,  time,  effort,  and  skill  and  very 
few  decisions  are  worth  all  that.  Moreover,  for  even  very  important  decisions  the 
formal  strategies  often  seem  too  coldly  intellectual  '. 

Observations  of  professional  decision  makers  engaged  with  on-the-job  decisions 
suggests  that  few  decisions  involve  e\plicit  balancing  of  costs  and  benefits,  let  alone  the 
use  of  behavionstic  probability  models  (Peters,  1979).  Mintzbcrg  (1975)  reported  that 
most  decisions  involved  only  one  option  rather  than  many  options.  The  decision  was 
whether  to  go  with  that  option  rather  than  a  choice  among  competing  options. 

Simultaneously,  the  evaluator  and  the  client  develop  judgments.  Their  ability 
to  form  sound  information-based  judgments  appears  to  be  grounded  upon  many  factors 
including:  r  * 

•  ability  to  reason  on  three  levels:  deductively,  inductively  and  abductively; 

•  ability  to  manipulate  overt  and  covert  information  normatively  and 
heuristically; 

•  ability  to  negotiate  or  at  least  share  perceptions; 

•  clarity  of  expectations,  both  internal  and  external;  and 

•  amount  of  cognitive  dissonance  and  uncertainty  of  the  decision  to  be  reached. 
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Out  of  the  judgments  represented  in  the  prior  step,  the  evaluator  synthesizes  a 
deiivery^yand  ccornpanying™ 

standard  variables  of  determining  the  b™"*™^a££h  thc  impact  of  priming, 
framing  of  the d^w  S^— n£L a'ctivatior  Jre 

ffit?^  ST  they 

JSU^iSe  likely  to  be  changed  if  the  outcomes  are  posed  negatively. 

oSXZX!^  -kers  are  required  to  integrate  ft™ 
oivLn  im.  iaui  i„d«miont  thc  quality  of  the  resultant  udgments  relies 

*  measurement  return  on 
m^mt  corporate  reputation,  and  contractual  expectations  for  projec  >; 
.  XSyTh  project  management  system  in  terms  of  yieldmgtimely, 
uSproioct  status  information  and  impacting  employee  morale  and 

.  ^oli^with  the  design  system  such  as  generating  information  specific 
to  design  and  development  decisions. 

While  the  research  base  is  lifted,  our  experience  suggests  that  clients  judge 
While  me  rcscar  familiar  (pattern  recognition). 

effective  operation  with  a  novel  strategy.  In  addition,  mere  seuro  i  "  ««-hhoff 
S£2WK.  further  ,he,r S^X";^ 
«™ng  a  var.ab|c  as  wt,n  research  (Weiss  & 

development  projects  consist  of  several  (sometimes  competing)  teams.  ™» 
^ialization  can  Rive  rise  to  functional  managers  who  begin  to  insist  on  making  all 
dSonfpertSng  to  their  respective  stages  and  inputs  from  other  teams  are  not 
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the  new  product  pending  written  notice  (Foshay,  1984).  This  sign-off  process  can 
dramatically  affect  the  openness  to  risk  taking. 

People's  cognitive  representation  of  judgmental  tasks  may  conceive  of 
"probabilities"  as  indices  of  belief  intensity  rather  than  as  ratios  of  favorable  chances 
(Cohen,  1982).  They  also  might  conceive  of  events  as  nonrandom  (Cohen,  1982)  and  fail 
to  take  into  account  all  the  potentially  relevant  information  or  all  the  potentially 
relevant  alternatives  (Hintikka  &  Hintikka,  1983).  It  appears  that  the  motivation  to 
respond  to  judgmental  questions  may  be  affected  by  at  least  two  needs:  the  need  for 
cognitive  structure  and  the  fear  of  invalidity.  Preliminary  research  by  Mayselcss  and 
Kruglanski  (1987)  suggests  that  revision  of  judgments  might  be  slower  under  a  high  fear 
of  invalidity  but  faster  under  a  high  need  for  structure.  "In  short,  fear  of  invalidity 
might  induce  a  tendency  to  be  conservative,  whereas  need  for  structure  might  induce  a 
tendency  to  be  excessive"  (p.  180).  For  example,  in  novel,  open-ended  environments  such 
as  instructional  hypermedia  projects,  designers  may  often  experience  an  acute  need  for 
guiding  structure  to  which  they  might  respond  by  adopting  inappropriately  high 
levels  of  self-assurance  and  a  closed-mindcdness  to  alternative  points  of  view. 

This  heightened  need  for  cognitive  structure  is  assumed  to  promote  an  early 
closure  to  a  solution  or  "cognitive  freezing"  (Freund,  Kruglanski  &  Schpitzajzen,  1985) 
For  evaluators,  this  suggests  the  many  risks  inherent  in  how  early  to  give  client 
feedback  as  well  as  the  nature  of  the  information's  significance.  Get  to  know  your  client 
cognitively!  ' 

But  there  are  risks  for  evaluators  as  well.  For  example,  high  structure 
eva  uators  who  rely  on  checklists  for  interactive  instructional  materials  revision  risk 
neglecting  some  critical  components  since  little  prescriptive  documentation  exists  at' 
this  juncture.  It  seems  more  defensible  to  involve  a  diverse  set  of  leaders  and  opinion 
makers  as  reviewers  if  the  goal  is  to  trend  acceptance  of  materials.  There  is  still  little 
to  guide  us  in  deciding  how  to  choose  experts  (as  well  as  evaluators)  and  how  to  guide 
their  task  or  structure  their  output  (Geis,  1987). 

A  person  with  a  high  need  for  cognitive  structure  is  likely  to  inhibit  the 
generation  of  competing  alternatives  to  a  given  recommendation  as  such  alternatives 
might  appear  to  threaten  his  existing  schema.  Previous  research  manipulating  the 
need  for  structure  in  such  ways  found  that  individuals  in  which  this  need  was  aroused 
tended  to  base  their  judgments  more  on  (1)  early  information,  rejecting  subsequent  data, 
hereby  exhibiting  primacy  effects",  and  (2)  on  pre-existing  stereotypes,  iTthis  sense 
being  theory  rather  than  data  driven.  Furthermore,  high  versus  low  "need  for 
structure"  individuals  tended  more  to  seek  comparison  with  similarly  minded  others 
likely  to  support  their  views  and  opinions  (Mayseless  &  Kruglanski,  1987).  High  "need 
for  structure"  individuals  were  characterized  by:  higher  initial  confidence  in  their 
judgment;  more  confidence  in  early  information  provided  by  the  search;  fewer  requests 
tor  information;  and  high  final  confidence  in  their  judgment 

Functionally  opposite  to  the  need  for  structure  is  the  fear  of  invalidity.  This 
motivation  has  to  do  with  the  desire  to  avoid  mistakes  in  judgment  in  terms  of  their 
perceived  costliness.  Where  high  need  for  structure  promotes Ta  freezing  of  the 
judgmental  process,  fear  of  invalidity  may  often  promote  unfreezing.  That  is  an 
increased  tendency  to  respond  positively  to  alternative  solutions  to  the  existing 
situation  and/or  an  increased  sensitivity  to  information  inconsistent  with  the 
prevaH  ng  order  and  negative  feedback.  Previous  research  manipulating  the  fear  of 
invalidity  found  that  high  fear  of  invalidity  individuals  suppressed  ^magnitude  of 
primacy  effects.  Instead,  they  had  a  tendency  to  translate  stereotypes  into 
discnminatory  judgments  (Kruglanski  &  Freund,  1983).  High  fear  of  invalidity 
individuals  had  lower  initial  confidence,  less  confidence  in  early  information 
requested  more  information,  and  had  higher  final  confidence.  Overall,  fear  of 
invalidity  might  induce  a  tendency  to  be  conservative. 
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example,  when  presenting  ^^^^/^J^SSi  with  negatively 
technology  in  a  I^IJ^^  ^  prevenHoTses)  are  mor?open  to 
framed  recommendations  (i.e., teKing  an  acuon  p -  W1  those 

Goitein,  &  Shapira,  1983  >•  „H„r*pH  ronromine  the  effects  of  standards 

simplify  the  rating  process  Naylor  Ilgen &  ^J^J™    validity  of  performance 

differential  weighting  into  their  appraisal  F^gme^ 

that  intuitive,  automatic  decision  ^"S/^X^Tve  of  interest  with  a  limited  set 

consist  of:  (1)  value  images  sutn  h     fa|     aspirations,  goals, 

vision  of  wh at  -  approve  C»t£^  m»g»  ^ and  ^ ts  used  t0  attain 
and  agendas;  and  ®*^t£«£Z£w  is  incompatible  with  these  images  it  is 
the  desired  goals.  If  a  decision .  /,  ^  acceotcd  as  the  final  decision  or  as  one  of  the 
rejected.  If  it  is  not  ■ncompahble  ^^^'^^  that  selects  the  best  among 

S^^Kw-l*lf  &  Smith,  1980;  Anderson,  1983)  and  control  theory  s 
"template"  (Lord  &  Hanges,  1987). 

=« =%£S3S5HS5Sr 

aUCrnatiThee contexlin  which  decisions  occur  appears  to  give  them  meaning.  In 
.HHition  oast  suaessess  and  failures  in  similar  contexts  seems  to  provide  guidance 
SSS^SSSS ^current  decision,  if  the  current  decision  is  virtually 
*£2Z  1  past  d«Jion  stored  in  memory,  it  is  considered  to  be  recognized  and 
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automatic  (Beach,  1964).  Framing  results  when  the  contexts  of  prior  similar  decision 
memories  are  used  to  go  beyond  the  information  that  is  presented  by  the  current 
Mtuahon ,  alone  and  interpret  new  contexts.  Thus,  we  might  expect  that  manipulating 
«  afS?  8-  P™£ess  ™uld  have  a  significant  effect  on  decision  mTking  and  choosing  f 
Anal  behavior  (Rachlin,  1989).  In  novel  contexts  where  the  exceptional  is  encountered 
it  seems  that  the  process  becomes  much  less  automatic.  encountered. 

The  decision  mechanism  is  influenced  by  whether  the  client  is  a  novice 
dec.s.on-nwker  or  expert,  whether  the  decision  environment  is  self-initiated  by  the 
chent  or  imposed  from  above  or  pressured  from  below  and  the  degree  to  which  the  client 
H   t °r  Tte  selection  among  decision  strategies  is  often  seen  as  a 

and  tnl  T™*  °<  °*n,t,VC  TCS°U™  (effort)  r^  <°  use  eTcSategy 

andthe  ability  of  each  strategy  to  produce  an  "accurate"  effect  (Beach  &  Mitchell 
1978;  Johnson  &  Payne,  1985;  Russo  &  Dosher,  1983).  <vmcneu. 

The  choice  mechanism  is  an  outgrowth  of  the  decision  mechanism.  Its 
efficiency  ,s  contingent  upon:   the  client's  perceived  self-efficacy;  the  environmental 
'"f  C,,ent'ssituation'-  ^  attractiveness  of  the  alteratives  and TnTen. L 

Z  tlSC'<  F?  eXPGnd,tures  of  effort'  ^ort  and  long  term  performance  valenccsTand 
the  type  of  delivery  strategy  required  by  the  decision. 

K«  c~  un?uy' outcome  and  implementation  of  the  ultimate  choice  results.  As  can 

SSTeSlSSS  mux""5'  dedSi°n  ^ Ch°iCe  C°8nitive  oT 

which  evaluators  must  be  more  cognizant  if  genuine  implementation  of  evaluation 

findings  and  recommendations  is  desired. 
Implications  of  Phase  4 

*«mnpnZr0         ^  °f  deCiSi°n  stra,c8ies  described  in  the  literature  arc 
compensatory  and  noncompensatory  models.  Compensatory  models  represent 
cognitively  complex  and  sophisticated  strategies  for  information  integration  (Einhorn 
k  Hogarth ,1981)  which  are  indicated  by  the  absence  of  the  interactive  u of  cues 
Billings  fc  Marcus,  1983).  Noncompensatory  models  are  indicated  by  the  in  erac  ive 
use  on^ormation  cues  in  which  a  low  score  on  one  dimension  cannot  be  compemated  bv  a 
high  score  on  another  dimension  (Billings  &  Marcus,  1983)  "mpensatea  oy  a 

Compensatory  strategies  refer  to  either  the  'linear  model  or  the  additive 
difference  model.  The  linear  model  assumes  that  each  dimension  for  a  decision 
fhln  h    H  ,S         3  V3,Ue  f°r  CaCh  altcrnative.  Comparisons  among  alternatives  are 
^teT^T^V  T?  '  V3,UCS       the  alterna'ive  with  the  greafest  value Ts 
u  I6  d,ffcrcnce  modcl  imPHes  that  decision  makers  compare 

^^rfrS  d,mCnS,°^       thCn  SUmmin8  diffcrcnccs  dimensions.  The 

summation  of  d,f  crcnccs  results  in  a  preference  for  one  decision  alternative 

In^dfr  y'    »9)-  WUh        HnCar  and  additivc  differcnce  models,  a  hfgh  value  on 
«  r,v°n  ^°mpenSatef  °r  countcraets  low  value  on  another  dimension  for  the 
n^^°n  altCrnat,,ve-  Noncompensatory  strategies  involve  the  use  of  s  mplitong 
If  £^V°  COmpleXity  °f  thc  dccision-  ™e  ma£r  noncompensatory  models  ^  ^ 
identified  by  Payne  (1976)  and  others  include  conjunctive,  disjunctive  texTcotraohic 
and  elimination  by  aspects  strategies.  lexicographic. 

Frequently,  clients  must  make  choices  under  less  than  ideal  conditions 

is  tvS7™PHhen  h3VC  inC°mpIetC  inf°mation  about  «*  S Tand  doubt 

Ind  PhPrIy  gCnera'ed  ,n  a  f™8  s,tuation  when  time  is  restricted  for  decision-making 
and  here  are  unanticipated  choice  points  have  been  presented.  Increasing  uncTrtaintv 

n£2£T!SS?  M      dCCentra,iZation  °f  an  organization's  communica^ion^cture 
interactive  instructional  technology  is  being  created,  from  aZdiS^Z^ 
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large  scale  multinational  corporations  and  how  the  ^  «J  "*2^C"" 

evaluators,  the  skill  of  its  leaders,  ana  tne  paruiuiai  ^  Xjf  .  „0  nonm 

1982       &  Slocum,  1984;  Schoonhoven,  1981;  Argote,  Etevades,  &  Melone  1990X 
1982,  rry& :  s oc     ,       ,       centralization  can  be  dysfuncbonal  given  centralized 
structure  irfom  m^  pooriy       decentralized  structures  for  complex  and  uncertam 
aTkfSSdCT  1972).  This  appeals  to  occur  because  members  at  the  hub  of 

Sc^hKout  ,he  group  rather  than  ono  member,  wc  expect  decentralized 
ZcSresTo  emerge.  While  this  projection  i,  fascinating  .o  contemplate,  tt  ts 

I  that  it  will  occur  with  much  discomfort  and  resistance. 
mMr*£XSc  medSto,  sua.eg.es  which  have  implications  ^'uators:  P-s 

„„,tlue  S  aSpira*nS  and  they  peK0.ee  that  there  is  little  common  ground, 
valuation  lets  the  parties  handle  the  dispute  on  the.r  own. 


^s^matic  combination  of  evaluation  and  cognitive  processes  can  guide  the 
and  he  vaSascd  approaches  all  lack  consistent  methods  for  ^t^aUzing  the 

program. 
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Figure  1.  Summary  of  criteria  for  three  evaluation  models:    objectives-based  (OB), 
decision-based  (DB),  and  values-based  (VB) 

MODEL  rRITFRIft  QB  DB  Yfi 

L  intended  outcomes  documented  •  # 

Z  unintended  outcomes  documented  % 

3.  document  contexts  leading  to  outcomes  . 

4.  document  processes  leading  to  outcomes  . 

5.  client  standards  overt 

6.  evaluator  standards  overt 

7.  client-evaluator  negotiation 

8.  program  improvement-oriented 

9.  hard  &  soft  data  balanced 


*  denotes  criterion  being  met  in  a  mode) 

x  denotes  criterion  being  inconsistently  met  in  model 
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I  Figure  2.   Polemics  of  an  Evaluation 
Questions:   To  what  degree  should  they  be... 
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Inputs  WrrmN  Contexts 
Phase  1:  Questions 

Information 

1.  Client's  Task  Demand 

♦  surface  and  depth  factors 

•  stable  vs  unstable  contexts 

♦  past  performance  of  clients 

•  client  belief  structures 


Figure  3 

Processes 


Outputs 


Perceptual  Representation 


2.  Evaluator's  Perceived  Task 

♦  prior  experience  of  evaluator 

♦  semantic/semiotic  content 
priority 

♦  confidence,  politics,  resources, 
audiences,  purposes, 
boundaries 


Action,  Response, 
and/or  Judgment 

3.  Evaluator  sets  Questions 
and  Standards 

♦  phases:  analysis,  design, 
development,  implementatio 


4.  Client  feedback  Message 
to  Evaluator 

♦  client-helpful 

♦  field-helpful 

♦  audience-helpful 

♦  expectations 

♦  spontaneous  vs.  systematic 


5.  Evaluator  Represents 
Feedback 

♦  effort,  rapport 

♦  match  mismatch 

♦  objectives,  management,  or 
perception-based 

♦  changes  between  task  & 
feedback:  overt/covert 

♦  formative  vs.  summative 

♦  internal  vs.  external 


6.  Negotiate  Paradigm  and 
Schema  Setting 

♦  negotiated  vs.  independent 

♦  response  certitude,  framing 

♦  self-set  vs.  assigned  standards 


Inputs  Within  Contexts 
Phase  2:  DESCRIPTIONS 


Processes 


Outputs 


7.  Evaluator  collects  data  8.  Evaluator  analyzes  data 


•  qualitative  vs.  quantitative 

•  stable  vs.  changing 
•indwelling  vs.  1-shot 

•  intuitive  vs.  analytical 

1  maximal/optimal/actual 
1  disjunctive  vs.  conjunctive 
'  additive  vs.  dicretionary 


•  information  load 

• description  vs.  judgement 

•  anticipated  vs.  unanticipated 
>  analysis  model 

1  audience  analysis 


9.  Evaluator  Reports  Initial 
Descriptions 

♦  feedback  time: 
simultaneous  vs  equential 
prove/improve 


10.  Client  Feedback  Message 
to  Evaluator 

♦  initial  certitude 

♦  credibility  of  Evaluator 

♦  match-mismatch 

♦  selective  perception 


11.  Evaluator  Represents 
Client's  Feedback 

♦  prove/improve 

♦  quantitative,  qualitative  or 
holistic 

♦  overt  vs.  covert 

♦  subjective,  intersubjective, 
objective 

♦  balance  of  positive/negative 
feedback 


12.  Evaluator  Edits 
Descriptions  and 
Reports  Alternatives 

♦  feedback  time 

#  framing 

♦  heuristics  vs.  algorithyms 

•  implications 


Inputs  Within  Contexts 
Phase  3:  Judgements 


Processes 


Outputs 


Information 

13.  Client's  Feedback  Message 
to  Evaluator 

•  expect  gains  or  losses 

•  truth  vs  utility 

•  Client's  certitiude 

•  purposes  &  resources 

•  selective  perception 

•  strategic  myopia 


Perceptual  Representation 

14.  Evaluator's  Editing  of 
Alternatives 

•  concrete  vs  abstract 

•  duration  of  events 

•  display  format 

•  self  efficacy 


16.  Evaluator/Client  Judgment 

•  inductive,  deductive,  abductive 

•  heuristic  vs  normative; 

•  confidence  of  Evaluator, 

•  expectations: 
internal  vs  external 

•  overt  vs  covert; 
proactive  vs  reactive 

•  uncertainty  of  decision: 
cognitive  dissonance 


Action,  Response, 
and/or  Judgment 

15.  Evaluator  Rates 
Alternatives 

•  feedback  processing  time 


17.  Delivery  Matrix  & 
Recommendations 

•  time  and  format  of  delivery 


Inputs  Within  Contexts 
Phase  4;  Decisions  &  Choices 
Information 


Perceptual  Representation 


Action,  Response, 
and/or  Judgment 


18.  Decision  Mechanism 

♦  satisfice 

♦  self-initiated  vs  imposed 

♦  role  of  Evaluator  and  Client  in 
decisions 

♦  risk  taking  vs  risk  seeking: 
error  penalties 

♦  novice  vs  exper 

19.  Choice  Mechanism 

♦  perceived  self-efficacy 

♦  environmental  demands 
(closed,  open  or  optimizing 

system) 

♦  elitist  vs  subordiante 

♦  attractiveness  of  alternatives/ 
incentives 

♦  past  expenditures  of  effort 

♦  performance  valence: 
short  &  long  term 

♦  press/compensate/integrate/ 
inaction 


20.  Outcomes  & 
Implementation 
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Table  1.   Criteria  for  Evaluating  Instructional  Hypermedia 

I  IntrrtflffiMfflY^^^  rcflcxive skiUs, expectations, motivation)? 

.  what  is  the  nature  of     ^J.^^Scor  Aeory  based)? 

versus  dynamic  culture  and  discoun*)? 
.  how  is  linear  and  nonlinear  JnfJ^  ™k£w  is  rclevant  from  irrelevant  information  filtered, 
.  how  is  content  granularity  developed  ( e.g.,  how  is  rciev.. 

chunking,  degree  of  modulanzauon).  structures  or  knowledge  structures? 

XSp—  in  diffcrcnl  modahues)? 

™  mode.  ""^XScally  and  referentially? 

.  how  do  lcamm  use  ^^mt^^lZ^S^xcmm^mbo^^i^Y>       _  „ 
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4.   Boundaries  h»n»,ffn  R«n»rf«  ami  ^flyjfigg 

•  what  methods  for  information  retrieval  are  Available? 
wSmtS^r1  fOTs^^"« hyP^cdia:  deductive, inductive &/orabductive? 

."        *bB  **■  P,annin8  match  execution  of  novice  and  expert  access  strategies? 
minhSTS^^ 

taSS^^  advisors  such  as  online  and  offline 

•  howsn^ycaTtn^ 

•  to  what  degree  are  metacognitive  or  self-learning  skills  overtly  laughf 

^wtat?extent «  heuristic  guidance  content  specific  in  hypermedia  contexts?  when  should  information  be 

:  «3SS!^^ 

5.    Boundaries  of  Indjyjdjal  WlTrlf 

•  to  what  extent  do  metaphors  emerge  that  help  us  conceptualize  this  comt>lexitv'> 

how  docs  hypermedia  influence  an  author's  cognitive  load  fe  /  can^i.^nlTil  h  ^      t.  . 
content  and  transitions)?  k     caPacit>' 10  makc  decisions  about  links, 

."  iSTiff^  f  uforJand  1,16  "»"■  P0™™ 11,0  significance  of  the  link'' 
reTtiS^ 

•  what  grammar  can  be  established  that  portrays  information  non-Iinearly? 
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ft  Willi™,,  1988;  RoM  arSo'T  1»T  W  1 W  rfj^,  "  <•  * 

course  of  the  lesson  ^liverXTB^BM^M^i^r^  S,tU*?^J  encountered  during  the 
l^Seidd.W.gner.R^^ 

Martin  J^^l^rt(l^^^^l  <1984>'  «d 

differences  in  the  efSS  ro,e  rf  ,carner  individual 

effectiveness  of  both  ratiomd^o^tto  or^S  the  ""P**  on  'earner-control 
highlighted.  Useful  instructionSR^  sUtes  of  ** ,carncr  ™ 

Recommendations forfuuSS ra£h«£ of fere? P  ^  ,nto account these variables. 
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Learner-control  of  instruction:  Overview 

assemble  and  present  to  different  ^S£S^^S  «  SiKof : feedback  andreview,  to  name  just 
inlformation.  amount*  of  examples  and [l^T^^^^h<S^^9ad  allow  the  learner  to 

inatructional  materials.  o 
preferences  and  needs. 

There  are  many  common  instances  of  computer-based  activities  falling  under  the  general  rubric 
of  "learner-control."  For  example, 

.     .standard  computer-based  ins^^ 

Thistvoe  of  software  follows  an  overall  instructional  design  strategy,  but  V^w****™™" 
mateSri!  -own dSons  about,  for  example,  what  topics  to  see  and  when,  how  many  exercises 
to  take,  or  when  to  quit  the  lesson. 

writing  skills,  mathematical  reasoning,  critical  thinking). 

what  learner-control  features  are  offered  (Duchastel,  1986a). 
.      on-line  computer  documentation  which  allows  the  user  the  options  of  either  foUowinj [a 
deSled  w^through  of  major  procedures  and  functions,  or  jumping  around  according £ the 
2cds  of  to  tSSS  TheseJ"hdp»  features  most  commonly  serve  as  simply  perfonnance  job- 
aids;  but  they  are  frequently  used  as  aids  for  learning,  as  well. 

And  as  the  newer  instructional  technologies,  both  hardware  and  ^r^m^rateinto 
instructional Stings,  the  possibilities  and  variations  for  including  learner-controlled  activities  in 
instructional  designs  will  likewise  multiply. 

The  first  arena  of  CBI  mentioned  above,  that  is.  traditional  ^mPuter-^'er^f.5r  *Kffl 

the  contusion  of  the  paper. 
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1>pes  of  learner-controlled  activities 


Intentionally  or  by  accident,  most  instructional  designs  end  up  consisting  of  a  mixture  of  learner 

^'^^^^■re^  ,"min«  «perie»c«-  l^nuszowski 
for  which  the  designer  will  need  to  make  decisions  about  the  source  of  instructional. contnd  the  n£ 
three  are  more  or  less  at  a  "macro"  level,  that  is,  apply  to  ^^S^SnX^^u^l^ 
units,  and  lessons.  (For  a  discussion  of  many  of  the  forms  which  instruction  car .  tlte  at"h?S  levels 
SSySJ  ^?£e)^■COntro,•  Bri*«8'  «■  1988;  Huff.  198^98? R^wnuiTl^ 

to  provide  ^^X^S^J^SS  rc«,mmcnds  consider  whether  and  how 

Prov1?*  learner-control  over  micro  instructional  experiences,  that  s,  those  learnina  activities  which 
occur  rntoa  lessons,  whose  durations  span  minutes  rather  than  hours  or  dayTlKS 

£57 ^SPoSIkS^TS^ SU<f  Spfmc "^^nicticaa even^dSwager  & 
Kojas,        Gagne\  1985),  the  bulk  of  research  on  learner-control  in  CBI  falls  at  the  "micro"  level 

«n,?CVeS  rc^hera  have  proposed  lists  of  computer-based  micro-instrucUonal  activities  which 
-hey  say  could  or  should  fall  under  the  heading  of  "learner-controlled."  Sternberg U984?fnr e*im3l 

study  and  in  what  order,  number  of  exercises  to  practice  and  their  level  of  difficult  mSado?  of 
review  or  supplementary  materials;  the  option  not  to  answer  questions.  ^SSSfftSTS^ 
made  optional:  amount  or  kind  of  feedback  to  see  following  practice  questi ions whe her"  to  exSe 

Uurillard  ( 1987)  presents  another  assortment  of  computer-based  learnina  strategies  which 

,  *!7  ~c  stuoent  to  skip  forward  or  backward  a  chosen  amount  or  to  retrace  a  route  through  the 

SS&ffiX0" 10 f "JT1  Whe° t0  View SUch features 48 '"dexes orconteiu^M  A  rather 
from^bh^Sw^f 'h  °f  1<S™er-COfltro1  <?  «p«  seq^nce  in  computer-bascdrstnictifn  cornet 
ZS^^S  Ji^Sf  f  8ys^m  "J8"^  ^  student' with  th«  «d  of  a  light-pen  and  T 
DSJeaoS SSZfSS^n  a.»e  10 P01"1  and^Ump t0 8ubt°Pic  in  the  ,csson-  ThisVpproach 
fm^tt^fashioT       env,ronments  ,n  which  ^ents  proceed  through  taanJK!  non- 

incl.„t^n^!.-Cr*teil0ry.P7i,ented  by  Lauri,lard  (1*7)  is  called  control  of  learning  activities  and 
includes  options  for  the  student  to  see  examples,  do  exercises,  receive  information  consult  a  d<SL 

Stei„0berX^e9tft,!ai,0^•  iT*  MoSt  °f  hcr  ,ist  of  >^3£  fi; 

btcinberg  s  ( 1984)  list,  but  Launllard's  seems  more  complete  and  grounded  in  theory        mc,uaea  ,n 

most  germane  sets  of  instructional  variable*  affecting  the  success  or  failure  ofCr<onttSTed  CBI. 
These  categorization  schemes  overlap  to  a  large  decree  and  differ  orimarilv  in  , 

Historical  background 

.  ^TJlf, idca  °f  lcamer-<:ontrol  of  instruction  has  n  fairly  long  history  which  wc-dates  the 

better  able  to  determine  their  own  instructional,,..^.  Campbell  (1964).  SS^SSS^SSl^ 
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contiol  over  instruction  is  at  least  as  effective    P^r^  inat ^J^^^/Sd 

the  program-control  students. 

Other  justifications  for  developing,  learner-controlled  or  "self-guided"  instruction  gre* rout  of 

own  pace. 

Bettor'*  ( 1987^  chatter  on  the  history  of  educational  technology  tracei i  many  important 

^4?  ^Kap^^  Sniined  five  distinguishing  features:  a  mastery  requirement;  self-pac.ng. 
student  proctors;  aVeliance  on  written  instruction;  and  a  de-emphasis  on  lectures. 

Most  current  literature  seems  to  favor  two  other  phrases  as  being  fairly  synonymous  with 

activities. 

Current  rationale  for  learner-control  In  CBI 

There  seems  to  be  several  philosophical,  practical,  and  theoretical  reasons  for  allowing  learners 

teachers  would  want  to  include  computer-based  activities  in  their  classes. 

On  a  more  practical  level,  Steinberg  (1984)  says  that,  if  learning  is  found  to  be  equivalent  in  both 
learner-  and  c^pE  controlled  settings,  design  costs  should  shrink  time  ^^A^^0^. 
rXed,Td  attitudes  and  motivation  should  become  more  positive  if  a  learner-controlled  framework 
for  instruction  is  adopted. 

In  his  instructional  design  theory,  Merrill  (1983)  proscribes  learner-control  of  content 
(encomrasm uSum,  lessk  and  module  selection)  and  of  strategy  (which  includes  various  forms 
of  w^SoS)  Sust  (1974),  Fine  (1972),  and  Bunderson  (1974) present .an asnrtment  of  learner- 
coHed  2  ti« derived  from  Merrill's  early  theory  (Merrill,  1973;  Reigeluth,  1979).  The  TICCX 
y^Sie,  Computer  Controlled  /nfoimation  Television)  system  provides  the  learner 
K^n^oMsome f  of  which  are  dependent  on  the  current  course  (such  as  reviews,  menus,  quizzes, 
SterTsto^  of  question  difficulty,  and  topic  surveys)  and  others  are  constant 

SSs  EJSinME^  *e  system  (such  as  backward  or  forward  movement, access  to  a 
Sato?  acS  to  a  glossary,  and  opportunity  to  leave  on  online  comment;  there  is  even  a  feature 
which ^vesTe Student  the  option  to  "CUSS"  at  the  computer  when  things  go  wrong!). 

R»i oeluth  and  Stein  (1983)  in  their  instructional  design  theory  also  hypothesize  that 

own  momenury  cognitive  requirements"  (p.  17). 
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Additional  rationale  from  a  different  perspective  comes  from  a  survey  of  adult  learning 
preferences.  Ppdland  (1979)  found  that  the  top  four  reasons  why  adults  prefer  learning  on  their  own 
were  expressed  as  desires  to '  set  my  own  learning  pace,"  "use  my  own  style  of  learning,"  "keep  the 
learning  strategy  flexible,  and  "put  my  own  structure  on  the  learning  project"  Discussing  the 
differences  between  adults  and  children.  Hannafin  (1984)  argues  that  under  CBI  conditions,  older 
students  should  realize  the  benefits  from  learner-control  more  than  younger  students  because  they  have 
acquired  more  (and  presumably  better)  learning  strategies.  ' 

a    .  Somee<f  *•  ******* ,fron»  the  psychology  of  basic  learning  processes  also  implies  possible 
advantages  of  learner-control.  For  example,  one  might  expect  a  learner-controlled  instructional 
treatment  to  induce  more  elaborate  mental  processing  from  the  student  as  a  result  of  their  pondering  the 
choices  wtth  which  they  are  faced.  Salomon  (1983, 1985)  refers  to  the  degree  of  such  mental  activftyas 

invested  mental  effort.  The  more  such  effort  expended,  he  implies,  the  more  mental  elaborations  the 
student  performs,  resulting  in  deeper,  more  meaningful  learning.  In  contrast,  one  might  not  expect  as 
much  cognitive  elaboration  from  student*  proceeding  through  a  more  "passive"  instructional  treatment 
In  plain  language,  learners  given  control  over  their  instruction  might  be  more  likely  to  think  about  what 
they  are  doing  because  they  have  to  make  choices  along  the  way. 

.  ( 19?5).  too,  in  arguing  for  the  need  for  more  attention  to  basic  psychological  processes 

7r^J^nJ,^nlm^  of,COIIiPl>ter»  00 1**™."*  "uPPwf  the  use  of  learner-control  of  instruction  as 
IsI^iJf  "ft ntS  to  *  -.*,op  $"ir  own  co«n,tlv«  "tructures.  That  is,  consistent  with  a  constructivist 
^2JK2flP  ?*u,8m°n' ^  JBT""  ** the,  'f arnin*  of  ComP,ex  knowledge  structures  is 
rwi  £5£  Winr  l«™er  himself/herself  can  participate  in  the  construction  of  those  mental  structures. 
This  construcuvist  approach  is  also  promoted  by  Salomon  and  Oardner  (1986)  who  suggest  that 
tti^^rfX**,  t ir  OWn  "V?!*™*  by  the  traits  and  goals  they  bring  to  the  encounter,  the  way 
Uiey  apprehend  the  technology  and  the  situation,  and  the  particular  volitional  choices  they  make  In  so 
doing.  learners,  particularly  when  given  interactive  opportunities  with  computers,  are  likely  to  affect  the 
way  these  opportunities  are  going  to  affect  them"  (p.  16). 

The  effectiveness  of  learner-control  In  CBI 

„„_   Unfortenately,  the  research  on  learner-control  in  instructional  contexts  does  not  support  its 
unconditional  use.  Many  authors  of  texts  on  CBI  design  caution  against  cavalierly  offering avarietv  of 
options  to  earners  (e.g.  Alessi  &  Trollip.  1985;  Jonasten  &  HannSm,  1987;  aS£!^SSSm 

S, ^5XfK   w-  u^iSCem  to 'mpr°.Va  ^enUI  ,earninf  0'Shea  •**  Self  (1983.  chap.  3)  summarize 
much  of  the  unpublished  research  on  the  effectiveness  of  the  early  TICCIT  system  and  conclude  that  it  is 
S'^Pf lte  widespread  use.  Merrill  (1983).  too.  concludes  that  college  level  students 
wf"nWn?       ft00*     °f  learner-control  options,  a  position  also  taken  by  Carrier  (1984). 
bnow  (1980),  commenting  on  the  use  of  learner-control  in  adaptive  instruction,  argues  that  far  from 

USSSf&l^t idual  differencc" on  ,curningt  pr°viding  ^^SSX  £23? 

mnt^iSSf"?1  °1  ,^e/-(*,ntrol  in  CBI  hM  typically  compared  learner-controlled  and  prognun- 
SSfSsS^S^^r^f1^  remini9cent  of  and  analogous  to  media  comparison  studies  fonducted 
in  the  1960  s  and  70  s.  The  following  is  a  summary  of  research  findings  comparing  learner-controlled 
computer-based  instruction  with  either  partial  learner-controlled  versions  or  coTplft  ^pS 
controlled  versions  for  the  three  most  common  types  of  dependent  variables  meas^uredXch  studies, 
namely,  learning,  time-on-task,  and  attitudes  and  affect.  ' 

,9enfra,,y  fP^1*  th^  studies  compare  treatments  which  present  a  mixture  of  the  specific 
ES^  DM  subject  to  learner-control.  Additionally,  contrary  to  the  stetemenK 
SSTf  n^1984)'  ^  adu  te  andLchi,drcn  ^  to  have  been  well  represented  in  the™ rtudies  Many  of 

have  beenSfn  clte^ 
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control.  Also  contrary  to  a  suggestion  by  Hannafin  (19*£  findings ^gjgSrt 
indicate  that  children  do  not  seem  to  do  any  worse  (or  better)  with  instruction  piaceo  unoer  weir  w» 
than  do  adults. 

A  few  studies  have  supported  the  use  of  learner-control  of  at  least  some  instructional  events 
/  a  a  S5 ,  1  o^Camnanizzi  1978.  Blermann  &  Free,  1990;  Kinzie,  Sullivan,  &  Berdel, 

HannafinJ.  (19B4i  suggestion  thatlcarncr- 
^TZ^'^r^^Ltl^g effect oTmemory.  Newkirlc ((973). tojgwM*** 
SSSJrSSng  b&it  for  leamef-control,  but  not  for  program-control  Mayer  (J^^Jg* 
^SlxSomes  were  learned  better  when  learners  were  able  to  control  the  order  of  prestation 

wWe^eouTc^ 

ffiX.^^JL5?»perfOT.  There  were  no  dilferences  between  these  groups  on  low  level 
S  to^er.Ta  pd  ^iale  task  investigated  by  Hlermann  and  Free  (1990),  studenU  who  «M.ld 
select  Ae  order  of  presentation  seemed  to  have  a  stronger  memory-trace,  implying  more  engagement  of 
cognitive  structures. 

in  contra*  studies  bv  R  C  Atkinson  (1972),  Belland,  Taylor,  Caneloa,  Dwyer,  and  Baker 
(1985)  Joh^andTe^yLn09ffl?.  Lee  and  Wong  (1989).  MacOregor  (1988a)  Momsonjtoss. 
in^RaiHwTnoS  Olivier  (1971)  Pollock  and  Sullivan  (1990),  Reinking  and  Schreiner  (1985), 
S^ra  Si™  feSSS  &S«n  (1980).  Tennyson  and  Buttrey  (1980).  Tennyson.  Park. 
^SSSS^^^i!^^  Christensen.  and  Hajovy  (1985)  all  found  various tjpeaor 
SfflS^T prSramSntrol  superior  to  learner-control  of  the  same  instructional  elements  fo  posttest 
SevlnUrffiyof  these  authors  speak  of  learners  not  having  or  not  knowing  how toi Uhze 
SropnTstrategies  when  they  are  leftto  themselves  to  manage  their  learning  environment. 

Interestingly,  most  studies  in  which  the  computer  controlled  the  rate  of  pac ing.  Jg/jU f  *nen8th 
of  time  which  screenfuls  of  information  were  presented  to  the  student  (Belland  et  al..  198V Dalton. 
lQ^ennvson  Park  &  Christensen.  1985;  Tennyson.  Welsh.  Christensen.  &  Hajovy.  1985).  found 
gSlESX^^  better'than  self-paled  conditions  (in  which  to^^^^SX*1 

atwhich  material  is  presented),  the  usual  fixture  in  most  CB1  programs.  One  study  by  Milheim  (1990). 
however,  found  learning  better  under  learner-controlled  pacing  conditions. 

Additionally  in  a  meta-analysis  of  10  years  of  interactive  video  instruction,  McNeil  and  Nelson 
( 1991)  conclude  that,  as  a  general  statement  program-controlled  conditions  are  superior  to  learner, 
comrolted I  They  suggest  however,  that  partial  (i.e..  "guided")  learner-control  over  review  and  pmcUce 
SvWeTniigntle  tS?  most  optimal  for  learning,  although  they  caution  that  too  few  studies  included 
such  conditions  to  make  the  conclusion  unequivocal. 

However,  most  studies  found  no  differences  between  Jf  amer-controlled  jmd  f££^n^n 
treatments  (Arnonc  &  Giabowski,  1992;  Balson,  Manning,  Ebner,  &  Brooks,  1984/1985;  Beard,  Lorton, 
Searte  & AtiSSSS  197?  Carrier,  Davidson.  Higson.  &  Williams.  1984;  Carrier.  Davidson.  & 
wS &m  ^%n^^nm\i»ml  ft  Kalweit.  1986;  Fredericks.  1976;  Goetefried A 
Hflnnafin'  1*5  Grav  1987  Hannafin  &  Colamaio.  1987;  Holmes,  Robson,  &  Steward,  198*  Hurlock, 
^t\  AM^^  l^^^B^^n,  &  Bessent.  1970;  Judd.  O'Neil.  &  Spelt.  1974a;  Kinzie  & 

Hur S 1976-'  Lahey  Hurlock.  &  McCann.  1973;  Lopez  &  Harper,  1989;  Mcann,Lahey.  &  Hurlock, 
"973 ^PriSre  &  991  Ross.  Morrison.  ft  o'Sell.  1988. 1989;  Schloss.  Wisniewski.  & 

cSwriahTTSa  Strickland  &  Wilcox.  1978;  Wilcox.  Richards.  Merrill.  Christensen  &  Rosenvall 
19Tw^5  ia^  9W)  The  various  conclusions  drawn  from  this  "ntxiifference"  finding  are  interesting 
andiend  toStTgood  deal  of  rational  izaUon.  Some  of  the  researchers  use  this  finding  to  support  the 
use  orieamrr-controT.  saying  that  programming  the  computer  to  handle  the  myriad  complex  types  of 
SranfhK^  in  a  tesaon  is  far  too  difficult.  So  therefore  since  their  research 

% ^oSd^eaaTdo  no  hum.  it  is  better  to  let  the  student  handle  their  own  lesson  blanching. 
oSe7re^archers  use  the  "no-differencc"  result  to  justify  program-control  of  instruction,  saying  that 


MD.  Williams 


7 

1089 


Learner-Control  Review 


other  benefits,  such  as  time  savings  (discussed  next)  are  realized  by  not  letting  learners  control  their  own 
instructional  paths. 

Time-on-task.  .Several  studies  also  included  as  a  dependent  variable  the  length  of  time  students 
took  to  complete  a  lesson.  Two  studies  found  students  in  learner-controlled  CBI  groups  taking  more 
time  to  finish  the  lesson  than  program-controlled  groups.  In  a  study  by  MacGregor  (1988a).  elementary 
students  worked  m  pairs,  and  those  in  the  learner-controlled  group  were  given  the  opportunity  to 
participate  in  an  online  instructional  game;  students  in  the  program-controlled  group  were  not.  The 
author  attributed  the  time-on-task  differences  to  the  fact  that  the  game  aroused  quite  a  bit  of  interest 
thus  generating  a  lot  of  talking  and  other  social  activity  within  the  pairs,  naturally  consuming  more  time 
in  the  process.  Dalton  (1990)  also  found  that  students  in  a  condition  in  which  the  computer  controlled 
the  pacing  of  materials  spent  more  time  than  those  in  learner-controlled  pacing.  He,  too,  suggests  that 
the  amount  of  socializing  observed  among  the  paired  members  of  the  learner-controlled  condition 
accounted  for  the  longer  time  spent.  Another  study  (Avner  et  al.,  1980)  found  that  while  students  in 
learner-control  conditions  spent  more  time  during  online  tasks,  they  spent  less  time  during  related 
online  tasks,  in  this  case  laboratory  activities. 

t  u     £  fW,^eSJPT,,!,°  ^fences  in  time  spent  (Hurlock  et  al.,  1974;  Kinzie  &  Sullivan.  1989; 

ey*        .1  The  bulk  of  8tud,es.  however,  found  that  learner-controlled  groups  spent 
considerably  ess  time  man  program-controlled  groups  (Fredericks,  1976;  Johansen  &  Tennyson,  1983; 

B^ttSy  1980)  erS*  ft      1988;  Tcnnyson  et  81  • 1980;  Tennyson,  1980;  Tennyson  & 

nom.  Rese^hers  investigating  "efficiency"  of  time  spent  during  a  lesson  found  mixed  results.  Dalton 
(HOT)  round  no  differences  on  achievement-per-time-spent  between  self-paced  and  lesson-paced 
interactive  video  formats.  Another  study  (Goetzfried  &  Hannafin,  1985)  did  find  differencesbetween 
groups  on  an  efficiency  variable  they  define  as  the  number  of  concepts  a  student  sees  per  minute  In 
jheirstudy,  learner-control  was  the  least  efficient,  that  is,  promoted  a  slower  progression  through  the 

e»  ,i  i  oSif?*  °f  ^S??  <Goe>XzSxi^  &  Hannafin,  1985;  Johansen  &  Tennyson,  1983;  Tennyson 
et  al..  1980;  Tennyson,  1980;  Tennyson  &  Buttrey,  1980),  shorter  time  was  also  linked  to  poorer 
Iierronnance.  One  possible  explanation  for  these  findings  lies  in  the  confounding  of  instructional 
SSJJi  JV^"^       am?unt  °f  'fractional  ^nal  seen.  That  is,  learners  navigating  their  way 

rT°  mi^pCnd '??  bme  beaP?r  they        to  skiP  over  lar«e  "mounts  o*f  instructional 
material.  If  this  omitted  material  were  crucial  for  overall  lesson  performance,  these  students  might 
.    y  be  expected  to  perform  more  poorly  than  would  students  progressing  through  a  program - 

SS^te  J?£r C?mp,ete"  Uppcr  (,985)  8U*8^  that  steH  SndTKer- 

S2Si2f^  T°  differing  amounts  or  kinds  of  instructional  material  than  students  under  program- 

iHS1^  (         f°ur  some  Sl}™*°™  't  •»  entirely  likely  that  each  student  under  learner- 
control  selected  their  way  through  a  completely  different  instructional  treatment!)  Therefore,  in  these 

SSS!Zt  Wf  e?er  thC  CU,pr?t  *°!  fai,ure  °f  ^nier-coitrol  is  the  fact  that 

S00^1,         or  "mP'y  MW  suboptimal  amounts  of  instruction  as  a  result  of 
£2L ^S^tJ  r     '  ^  *?       stud,es  do  nP°n  tnat  students  in  learner-control  spent  less  time 
SSJSL?  mstructtonal  screens,  (e.g.  fewer  examples).  This  issue  surfacesVgain  later  in 

wt,M»  /?SSllenlin|0n     Pr°blem  *  J*?"*?  choosing  low  amounts  of  instruction,  Higginbotham- 

22S?11and  *°?      ^f"80" (1989)  draw  the  elusion  that  learner-controlled  instruction 
fi  U,(l0nJy  ?,OW  ltuden^ to  xlect  ^te*1'  sequence,  and  presentation  style  variables,  and  should  not 
allow  students  to  choose  instructional  events  which  could  after  the  amount  of  content  support T seems 
safe  to  say  that  the  confounding  of  learner-control,  time-on-task,  and  amount  of  instrucHl materials 
a  matter  which  can  prevent  clear  conclusions  about  the  relative  merits  of  offering  control  to  learners 

As  with  the  findings  of  no  difference  in  learning  between  learner-  and  program-controlled  erouDs 

Sf1^  fhe  taUth°^  °f ?e         who  Shorter  time  for  Icamer^nirolX^ufr^ 
creative  in  the  implications  they  draw  from  that  finding.  Some  authors  claim  that  if  learning  is 
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ZSESSy  undesirable  outcome,  and  thus  should  be  avoided. 

MtUudesandAffecf.  Most* the^ 
computer-assisted  instruction  or  compared  with 

favorable  attitude  from  students  who  experienced  a J«^^^J«L.  more  ncgativVattitudes  in  the 
program^ntrolled  *™P^*  81  *" 


required  to  make.  , 

AfewstudiesCAmone&Gra^wsla  %  l^B^^^l^  £££  fe^XISSrea-"- 
1988;  Lahey.  1978;  Pridemore ,  A :  Klein 199 ) found  no  ^«"»  n stua 

between  learner-control  and  Pro^^J^^£ntriS^mproved  attitudes.  However,  it  is  quite 

SSS53S3S:---- , 

Newkirk.  1973).  I*«™f&^S^^^ 

videodisk  instruction  found  better  attitudes  toward  tJ«5™^?^31  of  lesson  pacing.  In  most  of 

control  pacing  compared  with  ^f^S^ff^  <W       MW  a 

the  studies  examining  attitudes,  students  were  only  expowd  to  on e  type  JJPJgg^  of  &  thc 

learner-controlled  version  or  a  P"*"**"^  attitudes^  dental 

studies  examining  attitudes  is  presented  %^^ac^n^^^°^tn\  versions:  completely 
students  in  Denmark,  each  of  whom  «JS6^^J*  SSK  SSSkwd  situations.  They 
learner-controlled;  partially  earner-c (f^J^S^a^S^<ool^.  Interestingly, 
found  the  overwhelming  majority  of  students  PJ^^^^iSKmS^  split  between  partial  and 
males  by  far  preferred  complete .leamer-c  ontrd  w^le  fema^pre  betwSn  gender 

total  learner-control  P^«^J^^1^n^.!32r  Sfically.  he  found  that  females  under 
and  learner-  or  program-controlled  treatments  ^^^SSkatad  toward  the  content  of  the 
lesson-controlled  pacing  ended  up  with  bette pacing 

lesson  than  females  under  ^ert^ntr°"^tKf n  ^^SSZmJ^d  lessons, 
had  significantly  worse  attitudes  toward  content  than  males  unaer  learner  ^ 

Some  reseat  invesUgaedU,.^^^ 
of  attitude  measure  called  caaMng  mol.w.l.0/1 i  (Maehr. htaSeSSS  •a'vily  at  a 

1987)  whieh  indicates  how  likely  a  "^XJ^JSSS&SSiS^'a  desire  to  leaJn 
later  time  without  external  ^^Sdeiits'  deaire  to  pursue  science 

voluntarily.  Kiniie  and  Sullivan  J'^^^^^tXarffoLvlW  learner-controlled  CAI, 

XanlS" L  iSSonlrol  over  progtanKontrol  on  conttnutng  mottvauon. 

Lastlv  a  few  early  srudies  investigated  the  effects  of  lea^-c^led  COTpuMl^ed 
mJSS'^L      <U ..tempos)  ^-^SSSSat  JB  C  <3R SASL 

^^ber.l^^nowev^.^1^  uJr^n  do  „  unifonnly  mrte  good 
M.D.  William,  »  Uamer-Conuo.  Review 

»  'uUV 


The  role  of  learner  characteristics 

madeU^^  ,984>-  F^w  of  these  approaches  haVc 

CBI  found  tartS  U^ffifl^  and  undcrstandability,  most  of  the 

interaction  within  aTSX  amines  such  an 

That  is,  students  ^el^vRIKSS^  v^IL?^  V"™*  1%P 
predetermined  instructional  features 1 1£ SSiSnSv  rf     V1,eWe* 38  pos8,b,y  intcracting  with 
learner-controlled  context Snow  1980?  o^„l«f  •  fly  e^t,Iv^eani,n8-  Particularly  within  a 

apUtt.de  rwalureT^      '     " |XW  "P"0"5  selecl0,s  according  to  their  scores  on  a  variety  or 
benefit  2^5S£S£ dmiSS J?  !°  [ind  '"•"■"•l  «*atme„,s  which  would  somehow 

Individ  differences  instead  oSS^e^ 
M.D.Williams 
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between  individual  differences  «nd  le«nung  tolhe  degiee^  IM^awwua  uiu 
outturn  would T«ed»in*.^ 

19&7a). 

m^ii  n  97«ft  discusses  several  assumptions  regarding  "aptitudes"  and  "treatments  in 

importance  of  learner-control  in  his  theory  of  instruction. 

That  is  while  ATI  approaches  seek  to  understand  the  differential  effectiveness  of  leamer-control 

venations  in  certain  ^JSCSSS  the  phenomena  of  moment-by- 

environments.  f 
Studies  by  Se^agner  ^ 

a^e^by^dS  llSs^rate  the  utility  of  variables  which  occur  dunng  the  course  of  learner- 
controlled  CBI. 

^teJSXracteristics  so  the  student  and  instruction  are  better  matched. 

So  both  person  and  instruction  variables  can  be  considered  either  stable  or  unstable  perhaps 

SSe  effects  of  individual  differences  rather  than  to  accommodate  them. 

tho  ™nH«.  "adaotive  instruction"  paradigm  presents  a  revised  set  of  larger  questions  to  the 

Z^t^^^'^^p tetween  ,eamer  differences  4,1(1  instrucuoa  undcr  801,16  d  g 

learner-control. 
Instructional  choice 

Learner-controlled  instruction  is.  by  definition,  instruction  where  are  ^«^"fc 

.wwinnTat  various  ooints  In  order  to  gu  de  the  design  and  use  of  learner-control,  it  is  necessary  10 
1%£Z$ th?compSlton  of  such  decisions;  i.e.  can  we  specify  the  precursors  and  effects  of  the 
decisions  students  will  make? 

Choice  is  at  root  a  psychological  phenomenon  determined  by  both  general  and  situational 
psycholo^ariablS  ASnjb.  a  student  might  have  a  proclivity  to  select  certain  types  or  large 
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ZT^^rti  iostructt?nal  pities,  generally  irrespective  of  the  specific  lesson  in  which  they 
may  be  engaged.  However,  the  particular  situation  may  moderate  the  general  tendency  For  examole 

SSsSfed  8Uf h  S?  opJL°"-  However' under  ceitain  con<«tion7,      student  may  perceive  specific 
instruction.  They  might  instead  elect  to  skip  over  such  materiaJ  because  it's  "a  waste  of  time  "  Both  of 

We  seek  at  this  point  to  identify  the  different  kinds  of  person  variables  which  it  is  coniecturad 
iSSSfSE  <hoice*  madc '0  ■•■  the  instrucUonal  material! £53&Q  £3pS 

account  for  the  unevenness  of  learning  found  under  learner-conlrolled  instruction. 

As  was  pointed  out  earlier,  Reigeluth  and  Stein  ( 1 983)  advocate  the  use  of  "informed  Icamcr- 
control  by  motivated  learners"  [emphases  added].  This  statement  suggests  two  wuSSSSSSSSSa 

Kcff         ffCrcnCe  Variabfe8  wWch  ^ innucnce  ,he  eff « venes»  SIS 

«».,^»xW«Jh?U,d  ft"* 1)6  intercsted  in  a  student's  capacity  to  make  rational  choices  (i.e.  an  "informed" 
student).   Rational"  means  how  adequately  they  can  appraise  both  the  demands  oHhe  task  and  ST 
aTd  P^no^.TAU°n,  ^ thaJ      in  ord" to       appropriate  instruction*  suSo^ennyson 
S££  SS?  <S'  ^i? 6  fstudent'8  "perception  of  learning  need,"  and  also  point  out  tne  nei  for  ite 

k£Z  S!iy  1  •ir.t°  **  °f  USC effeCtive  ,eamer-controlled  instruction.  These  percejions  of 
learner  need,  too,  will  vary  across  learners.  ^    ^  1 

k,  u  Seco?d,.X'be?a"?e.^0th  motivation  and  learner-controlled  instruction  are  at  least  in  out  defined 
by  choice  activities,  individual  differences  in  motivational  variables  might  iso oontribnte  tifS 
undmtending  of  the  differential  effects  of  learner^ontiolled  instructiSf  on  £33?  Wherefore  need 
to  ask  if  there  are  there  certain  characteristics  of  the  student  and  the  task  which  would  allow  us  to oS 

„r  »u  •  JhC  ™'icy/  Pj^ted  here  examines  the  relationships  between  students'  rationale  understanding 

Rational-cognitive  aspects  of  choice  and  learning 

Two  kinds  of  cognitive  traits,  prior  knowledge  and  ability,  offer  useful  Dossibilities  for 

ESSSSSTt*1*  nCgatiV;e  r?lU,tSu0f  Priding  learners  iidSS^BSffi^ 
2*  ~^,Sa!?,e^trol  .^th, achievement  and  with  ability  will  be  prcsen  Jin  tJm  toicthcr 
with  some  hypotheucal  instructional  prescriptions  which  could  take  advantage  ofThcse :  relsttoS^. 

1.     Prior  knowledge.  A  possible  explanation  for  these  findings  Droooses  that  individual*  Ho 
S5  Sw  *?TnS' but  ?thin  their  own  Perceptions  of  the  SStimtt 

^,dC  deC,S,°n  m  CS  ^S  vicw  su*8csts  that  an  increase  in  an  indiv  dual  Wura? I of 
SSSLiSS^^f^  ,nure,^ion  to  the  ,earnin*  <*sk  ^ould  result  in  their  making^o^ 
oT^th?^^5ivUlde,lt,l  "* ^  ^ expected  to ^ke  instructional  choices  wh?ch  *e  Sal 
only  to  we  degree  they  have  accurate  information  about  their  current  learning  state  This  su^esf^an 
approach  based  on  learner  prior  knowledge  or  achievement.  *  gg 

m-t^w1?  i?"df  °f  information  w.°"ld  students  need  to  know  to  make  a  more  rational  choice  of 
L  iSSr.^?  element  would  be  some  sort  of  estimate  of  how  S  one  ^Cws  and 
!Sm2     S neJnfed8  to  ,carn  Howcver.  there  is  substantial  evidence  that,  left  on  their  own  both 
children  and  adults  very  often  overestimate  how  much  they  know  about  a  ^wn^iStattel. 
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Landwehr,  &  Narens,  1982). 

These  latter  authors  use  this  finding  plausibly  to  explain i  wny  ^J^S^L  by  0«shea  &  Self,  1983, 
conditions  tend  to  terminate  instruction  prematurely  (e.g.  the  review  cX  TlLUl  oy  u  anea  « 

mentioned  earlier). 

capabilities  prior  to  their  taking  quizzes  on  the  lesson  material. 

The  notion  that  poor  P«rformera  «. <™£%«>^^^£^&l^ 

need  additional  instructional  support  tend  to  avoid  seeking  it. 

lilcir  cuEen!  knowledge "5*  activating  the  self-roonttonng  function). 

Sol  cTcSng or  SvXw.  to  the  students,  in  deciding  whether  they  need  more  tnstructton. 
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Addittonally^Steinberg  (1989)  suggests  that  instruction  should  gradually  wean  the  student  from  such 
crutches  in  order  to  promote  more  internalization  of  the  metacognitive  processes  Such  informaUon 

Hf2  tf*  tamBr,faU  W"  of  "deci8ion  •id«,,which  SS^bSn  shown to MS 

usetul  in  helping  people  make  judgments  and  select  appropriate  courses  of  action  (Pitz  &  Sachs,  1984). 

Related  to  thecase  where  students  left  to  assess  their  own  current  learning  state  is  the  case  where 
tins  assessment  is  performed  by  the  instructional  program  (a  "decision  aid'*)  duriHg  the  couree^he 
lesson.  One  would  also  predict  that  providing  students  with  updated  informaUon  £  to  ttS^oment-by- 
moment  mastery  level  would  improve  the  effectiveness  of  learner-control  over  providingTo^udf  * 

ImvK.(J^?'  1*1).and  Tennyson  and  Buttrey  0980)  support  this  contention.  Here  the  researchers 
S£S ^£S?nS±i^T^  ^th  such>f°rmaUon  and  show  beneficial  effects  oveTstldente 
K!!^^^11™'  fRe,at^  to,  ^  stud,es    •  rtudy  ^  Steinberg,  Baskin.  &  Hofcr  (1986) 

cK^tKd,ng  in  o™*wt  feedback  to  students  during  the  course  of  a  CBI  lesson  increSed 
the  chances  that  learner-controlled  memory  tools  would  be  used.  That  is,  students  were  able  to  use  the 
feedback  inlormation  to  help  them  decide  when  and  how  to  use  the  memory  tools 

...  .^"J8  &rtrn<*  unequivocal  however.  Ross  et  al.  (1988)  did  not  find  an  interaction  between 
student  selection  of  density  of  text  displayed  on  the  computer  screen  (high  and  low  deScsT and 
student  pretest  scores.  Additionally,  Goetzfried  and  Hannafin  (l^W^nSSS^St^ 
CBI Sting ment-by-treatment  intcraction  which  ™*  demonstrated  by  Rm^SS^^ff^  - 

An  additional  wrinkle  is  suggested  in  results  reported  by  Pridemore  &  Klein  noon  u/hn 
SKSaSS?"  2  Wf.1*  indents  under  two ^iSSSJ^^^^ 
in  the  elaborateness  of  informaUon  provided.  They  found  generally  that  students  in  the  less  elahorate 
condition  selected  less  feedback  than  those  in  the  more  elaborate  indln  ThS  suggesTtiS indents 
^SiSSSlSSS^  SUPP°rt  °nly  t0  'He  deg?ee  they  Perceive  it  will  help  them TKSSSSi then. 

&  T^^s^0^:^xa^^ on  their  ^ ved  but 

Instruction,  then,  designed  for  learner-control  should  have  as  its  eoal  the  exoarsion  and 

iSf  ?Ude?t,S  r "  PCrCCpU°n  °f  ™d       P»SSto^"f^£riy  for 

those  whose  are  deficient  in  the  accuracy  of  their  self-monitoring  ^  ' 

However,  it  is  not  known  at  this  point  whether  students  even  need  to  be  aware  that  self- 

S3  KSnS  tiiEEF^  -f^tion  as  a  type  of  Gamer' & 

Alexander,  lysy)  it  might  be  that  simple  directions  to  the  student  to  think  about  what  or  how  m..rh 
they  know  might  be  enough  to  dislodge  them  from  more  habitual  "mS^^^Uw^M 
somehow  activate  the  learners  own  untapped  self-monitoring  skills  it  is  sotcuTated  then  iTnS >i 
unnecessary  to  directly  inform  the  learneTof  their  master^ ^SS^SS^SS^l^ 

In  addition  to  supplanting  a  student's  monitoring  activities,  or  activatinit  existing  mnnimrina 

this  point,  metacogniuve  strategy  training  has  not  been  investigated  in  a  Icamer-contfff CBI  context 

nmviHii»  in^Z^V?'8''?  ^ tM^'  Another  explanation  for  thc  general  ineffectiveness  of 
5 t£o" n^eS?SSiJ?S  ,bC8mS  With  l!?C  SU8«Cstion  that  individuals  have  developed ^  cithe?good 
^Sfn^ff  r    •  efuhng  w!th  lcarn,n*  Problc™  Thc  metacognitive  scll-monitorine  processes 
™Sfd  earher  in  the  SCCtIon  on  War  JW/«fe*  represent  [subset  o  aScr  SlecST 
cognitive  processing  strategies  most  often  called  "learning  strategics."  k^ivSS^SXL  some  of 
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the  research  on  learning  strategics,  and  describes  four  classes  of  strategies,  all  of  which  have  clear 
implication  for  learner-controlled  instruction: 

•  Metacottnitive  strategies  are  those  processes  which  by  which  the  student  tells  themself  how 
mucSyknow.  It  Is  often  described  as  "self-monitoring."  and  reflects  a  sense  of  both 
knowledge  and  ignorance. 

•  Information-processing  strategies  make  up  the  largest  group  of  learning  strategies .  These 
stSegies  include  developing  readiness,  reading/viewing  for  meaning.  mgafflNgnrt. 
integrating  it  with  prior  knowledge,  expanding  or  elaborating  on  the  material,  and  finally 
levilwing  what  has  been  learned  These  strategies  seem  to  correspond  to  what  Merrill  (1984) 
calls  "conscious  cognition"  processes. 

•  Study  strategies  (in  the  past  been  called  "study  skills")  are  explicit  techniques  to  help  learners 
actively process  information.  These  consist  of  such  activities  as  note-taking  outlining, 
underlining,  and  the  identification  and  noting  of  patterns  in  the  new  material. 

•  SuDDort  strategies  relate  to  the  mental  climate  or  attitude  at  the  time  of  learning,  such  as  the 
dSfftSSSSt  can  internally  motivate  themself  and  stay  on-task  dunn gj^SSin 
Jonassen  (1985)  says  these  last  strategies  are  a  sine  qua  non  for  learning,  and  are  required  in 
order  for  the  other  strategies  to  be  effective. 

When  many  people  using  both  good  and  poor  strategies  are  averaged  in  a  study,  a less  ^^;jdtf^ 
picture  to  paTnS Sf  She  effectiveness  of  decision-making  as  a  whole.  Some  researchers  sugge*tf*  the 
use of theseof  these  strategies  is  linked  closely  with  the  concept  of  general  intelligence  (Snow  & 
Yatow  wSS  Ute  1S1 unreasonable  to  imagine  that  higher  ability  students  might  have  a  greater 
2^  rflJ^tSdSTiipon  when  fled  with  a  learning  problem  In  fact,  as  Snow  and  Yatow 
point  out,  very  often  the  concept  of  ability  is  equated  with  the  capacity  of  learn. 

If  indeed  we  can  infer  that  1)  higher  ability  students  consciously  or  unconsciously  bring  to  bear 
the  menJd  resources  appropriate  to  thefeaming  task  and  avoid  using  inefficient  ones.  2  ^J**9L 
stSdenSlomehow  eitheV  lack  or  don't  know  how  or  when  to  activate  their  learning  strategies,  and  3)  the 
sut^TEeTCSl  depends  to  a  large  degree  on  students  judiciously  app ttarMri 
reSurces  to  the  learning  problem,  then  we  can  begin  to  explain  the  mixed  results  of  learner-control  of 
iZESto Tas Wngto ^degree  a  function  of  learner  ability,  with  higher  ability  students  capitalizing  on 
learner-control  and  lower  ability  students  left  floundering. 

An  opposing  viewpoint  that  higher  ability  will  predict  better  learning  strategy  use  comes  from 
Clark  (1982)  In  a  review  of  aptitude-interaction  studies  he  first  hypothesizes  that  high  ability  students 
would  profit most  from  activating  or  cueing  methods,  that  is  techniques  which  J^l^ 
adopt  an  appropriate  mental  strategies  from  their  repertoire  of  stra  cg.es  for  a  given  problem  Second 
he  suceeste  that  low  ability  students  would  do  best  under  the  supplanting  or  modeling  methods  wh  ch 
are ^telEes  which  do  not  rely  on  the  student  to  use  their  own  mental  resources,  but  rather  exphcitly 
SdTSudent  through  the  optimal  learning  strategics.  But  regardless  of  what  they  would  need  he 
Est i  that  high  ability  students  would  prefer  to  choose  supplantation  or  modeling,  while  low  abdity 
sttidS  would  prefer  activating  or  cueing  methods.  Each  group  does  so  because  that  is  the  method 
wrcdvedS be  to  fowest  "mental  workload"  for  the  student.  In  this  case,  he  proposes,  neither  group 
Sects  an  appropriate  strategy.  In  support  of  this  hypothesis,  one  TICCIT  study  (Sasscer  &  Moore. 
1984)  found  that  when  students  were  given  the  option  of  terminating  the  lesson,  the  dropout  rate  was 
related  to  the  types  of  options  chosen.  These  options  patterns  were  typically  the  easier  kinds. 

Studies  examining  learner-control  have  found  some  positive  associations  of  ability  measures 
with  certain  SSTof f  cho£>  activities  in  learner-controlled  CBI.  Snow  (1979)  found  that  aptitude 
mSm* of  fiuid-analytic  ability  and  perceptual  speed  (in  addition  to  a  personality  variable)  predicted 
Z XicVactivhieTof Successful  college  students  in  a  BASIC  programming  task.  The  best  choice 
activities  were  described  as  indicating  a  reflective  and  thoughtful  style,  and  were  more  M<JBrt}y 
selected  by  high  ability  students.  However,  the  data  analysis  presented  is  sketchy,  and  contains  too  few 
subjects  to  unequivocally  trust  the  multivariate  analysis  employed. 
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Carner  et  al.  (1985)  found  that  between  a  measure  of  general  ability  and  a  measure  of  locus-of- 
^  W  1966),  the  best  predictor  of  amount  of  options  selected  was  the  ability  measure,  with 
high  ability  students  selecting  the  most  options  in  the  lesson.  Additionally,  Kinzie  et  al.  (1988)  also 
found  that  students  higher  in  reading  ability  selected  a  high  proportion  of  options  to  review  material  than 
did  lower  ability  students.  r 

There  is  also  evidence  that  ability  helps  determine  the  kinds  of  options  chosen  rather  than  the 
amount.  Carner  et  al.  (1986),  and  Snow  (1979)  both  found  near  zero  correlations  between  standard 
ability  and  achievement  measures     frequency  of  choice  of  instructional  options  in  computer-based 
concept  lessons.  Morrison  et  al.  ( 1992)  also  report  no  relationship  between  amount  of  instructional 
support  KkxUofis  made  by  students  under  learner-controlled  conditions  and  their  posttest  performance. 
Carner  and  Williams  (1988)  found  that  students  choosing  medium  level  frequencies  of  instructional 
options  had  the  highest  ability  levels.  A  study  by  MacGregor  (1988b)  also  found  differences  in  reading 
strategies  employed  between  low  and  high  reading  ability  groups.  In  contrast,  however,  Recking  and 
Scnreiner  (1985),  however,  found  no  differences  between  low  and  high  reading  ability  groups  in  any 
A&o f  °Pt,onB  ™"**£-  Examining  time  spent  as  presumed  indicator  of  on-line  strategies,  MacGregor 
(1988a)  compared  high  and  low  reading  ability  groups,  and  tc»<nd  that  higher  ability  students  spent  more 
55?  w*e"  under  Iwmer-control  conditions  than  did  lower  ability  students,  presumably  reflecting  the 
differential  utilization  of  on-line  strategies.  UK 

A  common  methodological  problem  in  studies  investigating  differential  options  selection  is  the 
conloundlng  of  ability,  pretest  performance,  and  posttest  performance.  That  is,  some  studies  compare 

°fflunf      .  ^  7i*?n  ,ow  Md  hi«h  Pretest  S^P8  <which  is  frequently  correlated  with  student 
ability)  and  low  and  high  posttest  groups  (also  confounded  with  ability).  These  sunogate  ability 
measures  might  limit  the  inferences  one  can  draw  about  learning  strategy  use  for  ability  groups,  but  still 
can  provide  insights  useful  for  further  research.  yp«^B»nai 

.  mi-  i  Fof  "S"!? ,e'  *rc,inaly8ls  of  dfta  presented  in  Seidel  et  al.  (1975,  p.  29,  Table  6)  shows  that 
while  low  posttest  performers  selected  overall  more  options  (supporting  Clark's  (1982)  hvoothesis 
mentioned  earlier),  high  performers  selected  proportionally  more  of  certain  types  of  optionVnamely 

S^SESS'Vliwfciiw  ^T*1  P"S  Wa5  not  the  for  RECAP  or  REVIEW  options  available 
In  the  lesson.  Cay  (1986)  found  high  pretest  students  more  "efficient"  in  their  on-line  time  (this  presents 
a  possible  contrast  to  MacOregor,  1988a.  where  high  ability  students  spent  more  time). 

There  is  also  evidence  that  ability  plays  an  important  role  on  the  attrition  of  students  in  large 
instnicuonal  units.  An  early  example  the  T1CCIT  system  (Merrill,  1973)  offered  college  students  a 
great  deal  of  choice  in  selection  of  both  content  and  strategy.  Results  showed  a  high  dropout  rate  but 

^n^^i^S^X^^)^  Wh°  ^  WCrC  8Cnerally  higher  »o 

The  finding  by  Tobias  (1987a)  discussed  earlier  in  the  Prior  Knowledge  section,  who  found  the 

SSSiA01*1^?  reV'.eW  SC,ected  by  studems  P^i'ivelv  related  to  the  student's  level  of 

pretested  knowledge,  also  might  be  interpretable  as  a  broader  difference  in  student  abilities,  as  well 

SSSft i3S^^aN,ily  fS  «  not  universal.  For  examp  SeT 

chaleT  Rubincam  and  Olivier  (1985)  found  a  relationship  between  pretest  and  options 

c«.„„iThf  mixf,.rcsu,u  froni  these  studies,  while  indicating  the  potential  for  ability  and  learning 

w?! XP  .1  °Vf  rt  Plrformance  nlearner^ontrollcd  CBlValso  demonstrate  that  more  research 
SSLtt  d,°£eU  U  'fJ'^'y  th^Vth?  8pe?ific  ^P*5  of  *****  Prescntcd  to  *e  ^udents  need  to  be  more 
2&  W"t the  "P^'fic  earning  strategics  it  most  conesponds  with.  Overall  abiSy 

smdln^fgUn'time16         *  d,ffcrentiatc  thc  more  rc,c™"  *****  strategies  adopted  b/a  given 

Some  types  of  instructional  interventions  do  appear  to  work  to  compensate  for  the  Door  use  of 
mental  resources  ,n  low  ability  learners.  Jonasscn  (1985)  presents  several  SigJtoKS!^,;^ 
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use  of  learning  strategics  in  computer-based  instruction  in  the  four  categories  listed  earlier  in  the  Prior 
Knowledge  section. 

Ability  appears  to  predict,  in  addition  to  the  individual's  perception  of  need  for  instructional 
support  (a  metacognitive  strategy),  other  types  of  mental  learning  strategies  in  which  the  student  might 
encase-  Although  the  relationship  between  ability  and  choice  seemr  more  tenuous  than  that  of  prior 
achievement  andchoice,  there  still  seems  cause  to  believe  that  appropriate  choice  strategies  can  be  made 
salient  to  the  learners  when  lacking  spontaneously,  perhaps  via  simple  infractions  or  suggestions,  and 
perhaps  by  changing  the  attractiveness  of  the  various  choices  to  be  made.  Additionally,  the  types  ot 
options  selected  appear  more  related  to  ability  than  quantity  of  options  chosen. 

Only  one  instance  was  found  of  a  learner-controlled  CBI  study  which  attempted  to  improve 
students'  strategy  use.  Elementary  school  students  in  Jacobson  and  Thompson's  (1975)  study  were 
given  prompts  at  various  points  to  help  them  make  appropriate  instructional  decisions.  Although  the 
instructional  treatments  used  in  the  study  were  quite  large  and  in  may  ways  not  comparable,  the  authors 
still  conclude  that  such  strategic  prompting  can  help  student*  to  make  appropriate  decisions.  Keigeiuth 
(1979)  proposes  that  learner-controlled  instruction  offer  students  an  "advisor  option,  a  sort  of 
prescriptive  "help"  feature,  which  would  suggest  to  the  student  various  so-called  "optimal  strategies  for 
how  to  process  information  or  what  to  do  next  in  the  lesson.  The  potential  flaw  in  this  proposal  is  that 
students  might  not  know  how  or  when  to  access  the  advisor.  Another  intervention  system  is  proposed 
by  Allen  and  Merrill  ( 1985),  which  provides  to  the  learners  varying  amounts  of  learning  •trategy 
suggestions  depending  on  their  aptitudes  for  accomplishing  the  learning  tasks.  For  students  of  low 
abilities,  for  example,  the  computer  would  provide  explicit  processing  representations  for  the  students  to 
follow;  for  medium  ability  students,  the  system  would  "guide"  the  learner  to  use  certain  previously 
learned  strategies;  high  ability  students  would  be  left  with  the  most  freedom  to  select  and  apply  their 
previously  acquired  processing  strategies  without  external  suggestions  or  interference  from  the  computer 
system.  This  type  of  system  has  not  yet  been  tested. 

The  idea  behind  all  these  approaches  is  to  promote  the  conscientious  and  mindful  use  of 
instructional  options  according  to  individual  needs  for  instructional  support.  The  following  section 
shifts  the  examination  of  the  rational  predictors  of  learner  choices  to  the  emotional  or  affective 
predictors. 

Emotional-motivational  aspects  of  choice  and  learning 

"Motivation"  is  a  very  slippery  concept.  J.M.  Keller  (1983)  defines  motivation  as  the 
"magnitude  and  direction  of  behavior.  In  other  words,  it  refers  to  the  choices  people  make  as  to  what 
experiences  they  will  approach  or  avoid,  and  the  degree  of  effort  they  will  exert  in  that  respect  (p.  389). 
Both  intuition  and  research  (Tobias,  1987b)  inform  us  that  poorly  motivated  students  are  also  very  often 
poor  performers  in  educational  settings,  too.  However,  the  derivation  of  instructional  prescriptions  to 
help  students  improve  their  motivation  to  learn  requires  a  much  more  detailed  exploration  of  both  the 
determinants  of  motivated  behavior,  and  the  effects  of  motivation  on  choice  and  learning.  That  is,  we 
need  to  uncover  the  reasons  (motives)  behind  particular  choices  a  student  may  make,  to  clarify  which 
variables  determine,  or  at  least  predict,  both  general  patterns  and  levels  of  choice  and  situation  specific 
choices  students  will  make.  Additionally,  we  need  to  investigate  the  reletionship  between  motivation 
and  learning.  In  path  analytic  terms,  both  direct  and  indirect  (via  the  actual  instructional  choices  made) 
relationships  of  motivation  and  learning  require  clarification. 

A  terminology  issue  needs  to  be  raised  at  this  point.  Many  researchers  would  argue  that  a 
"motivated"  behavior  might  be  based  on  rational,  logical  decision-making  processes,  and  thus  is  not  best 
described  in  terms  of  "emotional-motivational"  processes.  This  is  true  to  a  large  extent  (although  some 
could  argue  it  is  moot).  However,  for  clarity  sake  in  this  paper,  learner  "motivation"  refers  largely  to  the 
emotional  states  and  reactions  (and  their  consequent  overt  behaviors)  experienced  before,  during,  and 
after  instruction  which  have  an  impact  on  learning  and  choice.  So-called  "rationally"  motivated 
behaviors  were  discussed  earlier  in  this  paper. 
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A  large  body  of  research  and  several  psychological  theories  exist  which  attempt  to  describe  and 
explain  the  relationships  among  emotional-motivational  variables,  choice,  and  learning,  and  will  only  be 
touched  upon  here.  Instead,  the  implications  of  these  findings  about  motivation  and  learning  for  the 
design  of  learner-controlled  instruction  will  be  explored. 

/.    Achievement  motivation  and  learner-controlled  instruction.  The  history  of  motivation 
research  contains  a  sizable  body  of  literature  concerning  what  is  called  "achievement  motivation,"  in 
simple  terms,  a  person's  deaire  to  perform  and  achieve.  Because  of  the  behaviorist  tradition  from  which 
the  concept  sprang,  authors  on  the  topic  have  tended  to  present  "motivation"  as  a  fairly  broad  and 
inclusive  construct  defined  by  overt  behaviors  such  as  persistence  and  perseverance,  and  have  tended  to 
not  investigate  the  specific  underlying  emotional  states  which  could  be  said  to  be  the  sources  for  such 
achievement-related  behaviors.  Nevertheless,  their  research  is  still  very  relevant  in  our  discussion  of 
learner-controlled  instruction  and  deserves  exploration.  Discussion  of  learner  motivation  in  terms  of  the 
emotional  states  experienced  by  the  learner  will  follow  in  the  next  section. 

Following  this  tradition,  Lepper  (1985)  suggests  that  motivational  factore  could  operate  on 
learning  under  learner-controlled  computer-based  instruction  in  two  possible  ways.  First,  students' 
simple  exposure  to  instructional  materials  and  time-on-task  will  vary  according  to  their  motivation  to 
choose.  That  is,  the  often  repeated  failure  to  demonstrate  the  effectiveness  of  learner-control  might 
simply  be  a  function  of  the  fact  that  less  instructional  material  is  selected  by  those  students,  hence  they 
received  an  "incomplete"  lesson  compared  with  their  program-controlled  counterparts.  In  other  words, 
learner-control  ineffectiveness  would  be  totally  unrelated  to  the  learner's  emotional  or  motivational 
states  or  tendencies,  and  would  be  more  an  artifact  of  the  particular  set  of  instructional  events  they 
experienced  (or  more  likely,  did  not  experience).  This  is  a  similar  argument  to  that  given  earlier  in  the 
section  on  Prior  Knowledge. 

Indeed,  Ross  and  Rakow  (1981),  Tennyson  et  al.  (1980),  Tennyson  (1980),  and  Tennyson  and 
Buttrcy  (1980)  all  showed  that  students  in  the  learner-controlled  treatments  saw  many  fewer 
instructional  examples  than  did  students  under  program-control.  The  sheer  result  of  pooling  students 
seeing  both  low  and  high  amounts  of  instructional  material  would  certainly  be  expeected  to  show  lower 
overall  scores  than  students  only  given  high  amounts  of  instruction. 

However,  in  these  studies,  effects  of  the  amount  of  material  seen  were  confounded  with  the 
treatment,  learner  or  program-control.  That  is,  lower  amounts  of  instructional  material  were  inextricably 
linked  to  the  icarner-control  treatments.  Carrier  and  Williams  (1988)  experimentally  controlled  the 
amount  of  material  seen,  and  found  a  positive  effect  for  amount  of  material  separate  from  learner-or 
program-control  effects.  In  a  study  by  Morrision  et  al.  (1992),  amount  of  instructional  material  was 
controlled  for  by  having  two  program-controlled  versions:  one  with  "minimum"  instructional  support, 
one  with  maximum."  They  found  that  the  students  under  learner-control  actually  performed  poorer 
than  those  with  the  "minimum"  program-control  treatment. 

A  second  link  between  motivation  and  achievement,  says  Lepper  ( 1985),  is  more  direct  and 
related  to  covert  states  in  the  learner.  It  is  possible  that  a  person's  level  of  motivation  during  the 
performance  of  a  learning  task  affects  key  components  of  information  processing  related  to  learning  a 
position  also  taken  by  Salomon  (1983).  Emotional-motivational  variables  may  influence  the  direction 
and  intensity  of  attention  processes,  arousal,  depth  of  processing,  and  problem  representation.  Even 
though  Lepper  (1985)  points  out  that  many  of  these  information  processing  ideas  are  at  present 
hypothetical,  there  docs  seem  to  be  an  emerging  unification  of  the  underlying  mechanisms  1.  *ing 
mtHivation  and  achievement  (Humphreys  &  Revelle,  1984). 

Some  motivation  researchers  (J.W.  Atkinson,  1974b;  Brophy,  1983;  J.M  Keller,  1983)  posit  a 
curvilinear  (inverted  U-shapc)  association  between  student  motivation  level  and  learning.  That  is  both 
very  low  and  very  high  motivational  levels  can  have  dysfunctional  effects  on  learning. 

Given  this  relationship  it  would  be  interesting  to  look  for  interactions  of  level  of  motivation  and 
earner-  or  program-controlled  instructional  treatments.  Such  an  ATI  has  been  found  by  Carrier  and 
Williams  (1988).  Using  task  persistence  as  the  overt  motivational  index,  they  found  that  under  two 
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found  under  profnun-conlwl.J but »  mojly  I '"J^  ,rS^  hoSSver  they  only  reported  on  the  linear 
their  data  to  see  if  such  a  curvilinear  relationship  emerges.) 

This  interpretation  is  also  consistent  with  Salomon  s  (1983)  general  notion  ol  perceivea  oenuui 
characteristics"  of  instructional  treatments. 

Although  still  hypotheUcal,  three  instructional  factors  are  proposed  here  which might .be 
expected  o  inTrac?  with  a  person's  average  general  level  of  achievement  motivation:  learner  or 
program-control,  task  complexity;  and  extrinsic  motivation  variables. 

Learner-  and  program-controlled  treatments  might  be  perceived  by  ^S^^^Tm 

perceived  "ease"  of  learning  under  the  treatment. 

<inr„nA  fairiv  simDie  tasks  given  under  both  learner  and  program-controlled  treatments  might 

levels  of  moUvation  under  tasks  of  varying  difficulty  or  complexity. 

Last  the  object  of  using  extrinsic  motivators  would  be  to  try  to  increase  the  learner^  persi^ence 
rr  i^JISiin^mrticuSlv  for  those  students  with  low  motivation  levels,  through  instructional 

SSESJKT^  of  extrinsic  moav&u>? authority-  cot^Ttlon' 


wo  •  lllat:  *      i  u/  Atkinson  fl974a)  lists  as  examples  01  eMnnsit  hi^^^"  Vf  — -r-  

u^yshoidSlect  more  material  (based  upon  a  mastery  diagnos »  M  midt  "«2taKoSS?Il  al 
material  chosen  and  in  learning  equivalent  to  the  program-controlled  version.  Similarly, J Lamer  ei  ai. 

enmuraeements  within  a  leamer-controlled  treatment  did  increase  the  amount  of 

 A  1-  LAkAllt/\Mt 


on-task  behaviors. 


One  broad  orescriptive  framework  for  the  general  improvement  of  student  motivation  comes 
from  3  M  Ke lte7(1983  1987a  1987b).  He  presents  a  well  integrated  model,  which  subsumes  much  of 
&^  a!scS:  foTuie  JSign  of  novating  instruction  which  offers  prescriptions  consistent 
with  the  previously  mentioned  basic  research  on  learner  motivation. 

seaionatte^ 

^^^SSRS^S^SSm  the  natureof  their  perforn^ances  and  their  striving  to 
Srm  S^bcl tefs  to  taeU  model,  students  are  continually  forming  implicit  theories  about 
SSS^rttt oriei  IhSi  to  seek  particular  goals  related  to  confirmmg  these ,  (toon*  Dweck 
(1986)  describes  so-called  adaptive  (or  "mastery-onented")  and  maladaptive  (  helpless  )  i 


)  motivational 
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patterns.  The  maladaptive  pattern  is  characterized  by  an  avoidance  of  challenge  and  a  deterioration  of 
performance  m  the  face  of  obstacles.  Students  who  exhibit  an  adaptive  pattern,  in  contrast,  tend  to  seek 
challenging  tasks  and  the  maintenance  of  effective  perserverance  under  failure  circumstances. 

•u-  ^fi??* fo,lows  is  m  e*ample  ot  on  ?  avenue  of  promising  theory,  to  date  fairly  unresearched 
withiinCBI  contexts,  which  holds  promu  j  for  explaining  the  heretofore  mixed  effects  of  learner- 
comrolled  CBI,  and  suggesting  means  of  tnproving  instructional  designs  which  adopt  learner-control 
The  investigation  of  other  theoretical  framjworks  is  encouraged,  as  well.  In  all  cases,  however,  the  idea 
is  to  try  to  understand  the  nature  of  emotional  state*  the  learner  experiences  which  produce  healthy 
[adaptive)  or  dysfunctional  (maladaptive)  expression  in  terms  of  choices,  persistence,  and  perseverance 
during  learner-controlled  instruction. 

^  mfefr?rtion  of  Dw?$  ^d  fell's  (1988)  model  is  based  on  research  in  the  area  of  student 
attributions  of  their  success  and  failures.  Here  the  conception  of  motivation  becomes  that  of  a 
somewhat  unstable  factor  affected  on  a  moment-by-moment  basis  by  the  person's  perception  of  events 
happening  during  instruction  and  their  own  inferred  role  in  those  events.  Generally,  an  "attribution" 

Stu?i2S^i!Jl2^  *  P^fSSS?1         ?    ir  ?wn  ,ucce"  OT  f«",u«s.  Early  conceptualization  by 
Kukla  (1978)  and  Werner  (1974)  explain  that  the  degree  to  which  a  person  ascribes  the  causes  of  their 
own  success  or  failures  to  ability,  effort,  task  difficulty,  or  luck  will  differentially  predict  whether  or 
what  kinds  of  subsequent  performance  opportunities  the  student  is  likely  to  voluntarily  select  These 
tour  variables  can  be  grouped  along  two  primary  dimensions:  internal  versus  external  (analogous  to,  but 
not  the  same  as  the  familiar  "locus  of  control"  dimension  of  Rotter,  1966);  and  stable  versus  unstable. 

Other  researchers  have  recently  extended,  refined  and  reconceptualized  attribution  theory  For 

e!^P  'i  ?Vn^?n  ™d  ?me,i? (19^  1984b«  1985>  ^empt to  framc  ^d*™  attributions  in  terms  of 
emotional  states  they  imply  such  as  pnde.  shame,  guilt,  and  humiliation.  Additionally,  Dweck  and 
Liggett  (15*8)  present  a  model  which  seeks  to  explain  the  precursors  of  an  individual's  attributions 
along  the  controllability  dimension.  That  is.  they  attempt  to  explain  why  some  individuals  feel  more 
in  court I  of  their  performances !  outcomes  and  others  feel  more  "helpless."  These  developments  in 
attribution  theory  have  potentially  important  consequences  for  the  design  of  motivational  interventions 
during  instruction. 

™«».^iY^eW^tUd|eS  have  explicitly  examined  attribution-like  variables  in  connection  with  learner- 
contrtledCBI.  Treating  perception  of  intensity/  externality  of  reinforcement  (or  "locus  of  control  " 
Hotter,  1966)  as  a  predictor  variable  has  yielded  generally  unimpressive  results  in  differentially 
predicting  learning  under  several  instructional  conditions  (Tobias,  1987b)  and  in  predicting  overall 
choice  levelsor  learning  in  learner-controlled  instruction  (Carrier  et  al.,  1985. 1986;  Gray.  1989; 
Hannafin  1984;  Klein  &  Keller.  1990;  Santiago  &  Okey.  1992).  In  fact.  Lopez  and  Har£r(  1989) 
conclude  that  there  is  little  to  be  gained  by  further  research  investigating  Rotter's  locus-of-control 
constnict  in  connection  with  learoer-controlled  instruction.  Nevertheless,  these  negative  findings  could 
be  masking  potentially  valid  discriminations  within  groups  broadly  labeled  externals  or  internals.  For 
f« '     d,ffcrf nces  between  the  two  internal  attribution  styles,  ability  and  effort,  might  be  expected 
to  affect  options  selection  in  either  adaptive  or  maladaptive  ways. 

»nA„  JS?!?!!*1*  S?Jdy  <Fisher1et  al  •  1976>  treated  various  attributional  variables  as  dependent  variables 
under  conditions  of  learner-  and  program-controlled  problem  selection.  The  authors  found  that  subjects 
in  the  choice  group  made  significantly  more  internal  and  stable  attributions  during  or  following 
instruction  than  did  students  in  the  program-controlled  group.  They  also  found  no  treatment  differences 
d0en^bfU  !?"  v^ab  e  tthey        "control-no  control ;"  but  they  do  not  provide  ^rational 
definition  of  this  variable  to  aid  interpretation.  Additionally,  even  though  these  researchers  did  not  take 
baseline  measures  of  attribution,  nor  plot  the  changes  in  attributions  occurring  over  time,  their  study  still 
supports  the  short-term  modifiability  of  attributions  as  a  possible  result  of  treatment  variables 

™„tmi  VSaH^  ?  nler'?  i4*CS  m<idc! .(1983' 1987a'  1987b>-       attribution  theory  and  learner- 
con  rol  would  potentially  play  a  useful  roles  when  attempting  to  improve  student  Confidence  In  some 

Sff  i  »b £ntS  might  receive  attribution^  feedback  to  enhance  the  feeling  oVXT 
they  can  do  it.  Additionally,  they  could  be  given  some  degree  of  control  over  their  learning  Situation 
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Snuuon.  under  ^^.^JSffiflc  tanSSSnSK.  as  optaul  pntctice  item.. 

However, J.M.  ^f^'^, ^^^i^^^T,,  „ 
which  .hould  be  offered  to  -udentt.  end        ™°°f  "Suoatteory  for  explaining  lie  nuxed  effect. 

iSuonS  deSgn  atrateglea  which  Incorporate  the  theory. 


instructional  oeMgn  »wiu*|iv«   — r 

M.niPu>«U™direc..d« 
classes  of  inatrttctionnl  0 "»"»"" *  "mmenrUtion.;  aJ  3)  lho» 

or  failure  genertded  by  Inatrnclion. 

in  the  flrs,  «**  d.  are  included  «yJ^B&T2TCCT|ta 

*     f  ti  ATI 
^AddLallyinUtisc^ 

iSSSsaSSSSSSSS— -'•^ 

AisoinUusclassaretnanipuladons^^ 

Suggestion,  of  nonnreferenctng  C^>«»^'«^^^3!»be  compared  lo  others,  or 
Examples  would  be  general  ffnSSe  on  a  particular  task,  bow 

""^T^XLatosu.^^ 

dealing  with  micro-instrucUonal  adaptations  of        W^JJ^  such  as  those 

S are  forward-looking,  and  ^JS^^^SSS^S^  might  include 
menUoned  in  the  earlier  section  on  "tavernM  ^f^J?™^  difficulty  level  depending  upon 

Snt-s  a'ttribuUor.alstyte.  but  again  on  a  very  local  level. 

•n  the  M  class  of  in—l  T^^SSw^^^™™'*.** 
imerpretive  comments  on  a  sludenl  V pf«mMma«W Series  is  to  intentionally  alter 
task.  The  goal  of  these  refkettve  or  backward-  ^'"«STot  tSLd  enough,  or  when 
artributionf.  ComnKnttto^ 

SfSnal  paUerrJin  the  «ufenU  would  be  even  more  fnutful. 

A  doctor!  di«rt^»  conduct  by  ^.^^S^S^TwSn0' 
attributionally  related  feedback  on  ^fiH3f«5»»  in  the  stud;'  was 
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2SftS5JSr*     *  ^'^'Sttiiaa ,986; 

condiut^M^ 

Kbit*  dysfunctional  attribiSSSes  out  SSidtaJ^S? Snf  ""proving  learning  for  who  normally 
on  those  who  otherwi« S KSTiE  Jtf.K^  stCrras,P  taw  *  *l««ioi»  effect 
umler  learner-control  SSSS&i.i*'!*,1*1  tt^>i  »ve» to  a»<te'" 

CBI  JhSi&tffi^i;",,l  inflective**,  of  leuner^onMled 
motivational  tendencies  inch b  i^KSSIS^  ^  learners  have  acquired  maladaptive 
perciate.ce  o,  rSS^^^^^^^i^0^         («*.  Sowing  low 
one  particular  motivational  IheS^StR  «!5SSSffiJ^    L**™,  "llhough  scant,  that 

motivational  behaviors  for  learSi  Sn  T  J"  eXplSle<l  10  '"I*0™  "»  «•-<■■* 

motivation  to  achieve wtemlSS^^S  The  '"^ ls  "> "«»»"  >»* 

heip  the  student  to  optimt*  mei7sX™  of  *SiSS«££S?  ^  m  —  •» 

Summary 

based  Js^ffiwiS  SSSSSSiE^  of  learner-controlled  computer- 

leamer-control  ■upoiJtop!5SS^^fc^  ^jSnS™  ^"X*"5'  prcdictcd  to  show 
have  been  disappointing.       *,,un•co^l^o,  However,  empirical  findings  related  to  these  predictions 

responsfble1^  ?  of  mediating  factors  were  likely 

were  not  «p.blXKSS  of  £  SS5"S?iit  WM  ^  ll?at  »mply 
difference  variables  were  suSeVtedtc  I* S  ft*??  ?,VCu  Two  ,ar«e  Agones  of  individual 
oriented  variables  and  e™ioS^  thcse  student8:  rationally^ognitively 

under  le^rneS^  to  predict  student  success 

estimate  the  amount  of  instroSa I  stp^rt  tty  wouW  neS (St^^  °f  the  8tudcntS  to 
knowledge  were  not  able  to  effective,  v  S y.  w.ouia  necd-  (Students  with  low  amounts  of 
viewed  «f  tth^^^f^^^^^J^f^)  AddiUonallv,  student  ability  was 
problem.  (Low  ability •S^pSSSSSSS^ SZtSP**  £bearwhen        with  a  leading 
them  that  higher  ability  aSnte  &  S2?     rePertoire  of  learning  strategies  available  to 

differences  withteHe^oSed  in^tmcS8  U°nS  ™  ^  f°f  "^nmod&ng  these 
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Suonal  selNpercepUons  and  hence  inore  functional  behaviors. 
Recommendations  for  future  research 

i    What  -.necific  instructional  events  are  most  or  least  amenable  to  providing  or  vvithdrawmg 
leamer-L^M 

designers  may  draw  upon  to  build Jej^^ 
settings. 

opUmal  processes  and  perhaps  attempt  to  alter  or  at  leas  steward  &  Whitfield,  1988)  but 

7c  la?k  of  comfortwith  suet  methodologies  by  learner-control  invesUgators. 

3   Related  to  the  previous  suggestion,  it  is  time  investigators  more  closely  examined  the  social 

SjS  ?2£C7££         S  a.  the  ^  Rather  ton  at.cn>,*  <o 

learners  can  feed  off  each  other  during  instruction. 

a   i  -am^r-control  shoud  be  much  more  closely  investigated  under  other  common  or 
develot^n^s ^tSSX^^^«^^  such  M:  simu,ationsi  hypemiedia  and  other  onlme 
dSffl  (sffi Teffiirenlyclopcdias);  online  help  and  other  ff  «o^ 
SSl  All  of  these  contexts  (possibly  except..*  ^ff£SKS& 

S^rese^ch  "oLuctedto  sort  oui  the  peculiar  learner-control  factors  m  each  wh.ch  need 

attention  or  support. 

5  There  needs  to  be  a  greater  link  make  between  learner  -controlled  CB I  research  and  a 
growing  body  (If  Eture  on  the  topic  of  self- regulated  learning.  Bnefly.  this  research,  developed  by 
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McCombs  and  associates  (McCombs.  1982. 1984;  McCombs  &  Marzano.  1990)  and  bv  Zimmerman 
a^a^.a^prninerrnan.  1990;  Zimmerman  &  Martinez-PouTST 988? 990)  c^nSSSes 
student,  as  "metacogmuvely.  motivationally.  and  behaviorally  active  participanu ihV iSSSn^nt 
S"™  Z~& Martinez-Pdns.  1988  p.  284).  Although  CiifiS* 
SuE^^«????!  00  *  "J"  1?acro  lcvel  ^  '"WtaJ  strategies  occurring  during  succesXilf- 
regulated  earning,  this  literature  has  clear  implications  for  the  inclusion  of  motivational  vSto  in  the 
design  of  learner-controlled  instructional  systems.  In  fact,  some  investigatoreSvV mriTbenm  to 
explicitly  address  motivational  variables  op-TOing  in  self-regulated  Sunerafc V ScS :  1990b 
Ztmnennai t&  M^nez-Pbns.  1990).  Ad&tionally.  there isteginni^ngTemerSe' u interest 


Conclusion 


!KL  if  ^ 18  ?°*  M,hi«y  y  Prescriptive  or  determined  or  wmplicatedlsSchina  or  aSLr 
2teSS^fer:S«i,nd ^P"^-^ approaches.  iM^SSdt?^ 
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Introduction 


™_,.,T.he  'as' •yeny-nve  or  so  years  have  shown  an  explosion  of  research  and  develonmem  in 
comp^ter-asMled  and  computer-managed  learning  environment*  AcXcaTvSS  tfie 

Misanchuk*^^^^  StUdent  durin* instruction  **  myriad. 

usage  attoJ&^^S!^  ^!to«*uk  ■*»  Schw>er  (1992)  is  to  collect  such  data  for 
Sns^eX  Ja£v?  *"  COmbi"es  °f  b°*  cf  Eta  ,wo 


evaluati™  ™  InSfif  inv.<?iS;>,°r '?  '  v***™  developer  concerned  with  the  formative 
Svt^o^^ 

which  in  wy  SHi^te"8'?^?*11  instructional  system 
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Essential  User  Information  to  be  Captured 

bsSSSSSESSHSSSSs- 

experience: 

1  the  identity  of  each  user  interaction  with  the  software^  This  information  answers  the 
question.  "What  action  did  the  student  take?"  and  may  include: 

•  mouse  clicks  on  buttons,  fields  and  cards 

•  responses  typed  i  nto  the  keyboard 

•  menus  pulled  down 

2  ...thespecificscreenlocaUonofeachmouseevent.  This  data  answers  the  question, 

"  Where  did  the  action  take  place."  and  includes: 

•  where  the  mouse  was  clicked 

•  where  the  cursor  was  when  the  mouse  button  was  released 

i      the  time  at  which  each  learner  interaction  occurred.  From  this  data  can  be  calculated 
i^^fiSSS^!?wSK«  sometimes  called  "latency."  mat  is.  the j»o  spent 
SLKuonsof  the  learner.  The  time  taken  to  understand  a  screen  «to^W«a 
lotion  can be  used  to  study  the  learner's  cognitive  processing  (e.g.,  see  the  work  of 
Tennyson  and  associates). 

I.  journal,*  terms,  these  What,  Where  and  When  ^J^^f^^SSt 

nSSm  and  designers  while  the  Where  will  become  less  relevant 

What  follows  is  an  in-depth  presentation  of  a  particular  set  of  embedded  <»£puter-based 

systems,  as  well. 

fftlTf  tVIT  Pf  HyperCard 

The  examoles  that  follow  are  developed  in  HyperCard,  a  tool-building  environment  for  the 
Macinti^ 

environment  on  any  platform,  including  Autiiorware,  Plus  Director, ,  stroc7ure  may 
To  clarify  the  descriptions  of  programming  logic,  some  explanation  of  HyperCard  s  structure  m  y 

be  useful. 

u„~»rrarH  k  u«5ed  to  create  stacks  which  are  collections  of  individual  canfc.  Cards  can 
shareco^Skl,^^ 

^^^^^^^^^^ 
button,  or  the  pressing  of  the  Enter  key  within  a  field. 

ProaramminK  in  HyperCard  is  oriented  around  these  objects.  Unlike  ^environraenls 
in  ^c^lS^il^ncd  in  a  single  program  listed  all  in  one  place,  a  HyperCard  stack 
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?£K??n  T1^  P^?""?8  in  form  of  handlers  within  individual  buttons  fields  and  cards 
The  handlers  for  tracking  learner  actions  are  described  in  the  next  section 

Explanation  of  th,  H.nHi,n 

There  are  four  major  handlers  needed  to  track  learner  actions  in  aHyperCard  stack: 

•  S/^S^1  -.linitiJaiize8     trail  variable  which  stores  each  action 

•  MOT oTrcul  -  adds  a  specific  event  to  the  trail 

•  SaveTmilFile  -  saves  the  trail  variable  to  disk 

•  ShowTrail  -  shows  all  mouseclicks  accumulated  in  a  given  trail  file 

AfUTQTraL  Once  the  Trail  variable  has  been  set  ud  this  handw  ct™**  h.h  ok~„*     l  , 
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on  mouaaUp 
AddToTrall 

visual  chackaitooard  slow 
go  cd  cradlta 
and  mouaaUp 

"Sayonara"  Button  Script 


on  mouaaUp 
AddToTrall 
and  mouaaUp 

Background  Script 


Sa^MEii^  When  the  stack  is  closed,  this  handler  takes  the  Invariable  from  memory,  adds 
JSSKnd  total  elapsed  time,  and  stores  it  in  a  text  file  on  disk. 
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Smmk  An^it  Trail 

Slack  opened  at  8:25:25  AM  on  Satunlay.  January  9, 1993 

i«  ?2,Sird.id  ,0927,.«f<»  button  "Nexf.29.53.255,31 1.259  313 
?' ^'«£     w""1  bUI,0n  "S,udy  Hi™««na-.3.65.3973939739. ' 
2  «2  u^*  ^e?u>c*I<! buMon  "       Hiatorical  Sturr.9.35394\39397.62 
5»£ £"^1  T  J™'  '  ^-.8.88.447.202547504. 

button  Take  a  Look-.7.85.445.20 1.446.202. 
w  .,0ncal,bkind  bu,ton  "Me"u".3  07.43.283.45.287 
a Menu;cfrd  button  "     °""»  Here-3.05371.239371.239. 
8.68.35.poice.card  button  "Sayonara\2.97.40.305.54300. 

inT^l^f"^,*  bu,t0n  "Quit".22.4237l.203370.203. 

S^'SS^gS^I  ^  "^"•013-37^203363.203.Me„u  item:  ouit  hyperc^ 
Total  elapsed  time  =  91.20  iccondi 

QQOQQQ 

Event  Number 
Elapsed  time  In  seconds  since  the  stack  was  opened 
Card  on  which  the  event  occurred 

I     The  target  of  the  mouse  click 

.     *  i 

5, 59.27,  KanJI,  card  button  "Take  a  Look",  7.85,  445,201,  446,202 

Elapsed  time  since  last  event^       f  ^ 
Location  of  mouse  click 

Location  of  mouse  release 
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Analyses  of  Audit  Trails 

often  times  overwhelming  volume  of  data  generated. 

a«  csm  be  seen  from  the  previous  section  on  how  to  collect  audit  trails,  if  the  investigator 
wishes  tocS™ 

make  sense  of.  This  is  because  clata \ox  P™'^*  ^,le  to  standard  spreadsheet-oriented  data 
students  do  not  have  a  consistent  or  parallel  lormai  amcnaoic  mj  Cdcc      «  a  ^  Rarelv  is 

S  E£t^l£tt2S&&  £&-  d.  same  tafftttari  even.) 
prevent  adequate  staUstical  interpretation  of  the  data 

As  it  turns  out,  there  is  no  "right"  way  to  interpret  such  data.  In  general  ^ugMiere  is 

da^ShTS 'SfilJi  into  summary,,  averaged,  or  otherwise  collapsed  variables. 
Finally,  some  pertinent  measurement  issues  will  be  discussed. 

Q..,.lltatlve  fn«.ivs»s  of  original  (non»agpr<gated)  data 

One  option  for  investigators  is  to  not  even  attempt  to  aggregate  the  data  into  reduced  sete  of 

SffcSS SSSoMA  lall8*.  is  usually  required  for  such  analyses  be»»eAae  are 
seldom^nough  resources  available  to  conduct  analyses  on  a  large  number  of  students. ) 

a  nationally  researchers  may  want  to  adopt  qualitative  methods  to  examine  each  student's 

here. 

Rather  than  attempting  to  interpret  student  responses  by  simply  staring  at  the  audit  trail 
playback  routine. 
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Illustration.  The  ShowTrail  handler  was  developed  to  reoresent  vi<!iiaiiv  th*  hicfrt„,  ~r 
mouse  clicks  recorded  for  one  or  more  users  of  a  stack  TtakS?  fS mi  ^!L  f,  °f 
round  button  on  the  location  of  every  mo^click  stored  '  ^  "*  ^  placcs  a 
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tu,  and  hu  (pronounced  like  the  English  words  "sea- 
tea",  "two",  and  "who")  do  not  exist  in  Japanese  So' 
they  substitute  the  sounds  above.  Go  ahead  and  click 


This  allows  one  to  see  several  things  at  a  glance: 

•  Which  buttons  were  clicked  most  often,  least  often 
Sno  feTult00     SCreCn  WCre  misinterP'eted  to  be  buttons  and  were  clicked  on 

•  What  port  of  a  button  was  clicked  on. 

otheis  dicke^^Z^S^c^U^       «  *»  pom  to  relura  to  the  menu,  while  the 


others  clicked  the  forward  arrow  to  continue 
Quantitative  analyse*  Af  aggjsgflkd  rffltf| 


investiga^^^ 

to  collapse  data  into  lim^S^^yU^L^^'  **  m?«W*  want 
might  want  to  average  the  studen?Sr  for1analys,s-  For  example,  the  investigator 

count  the  numbefo^timcs  tSSUSXSZ^  *  feW  typCS  °f  e,VentS;  or  W dimply 
aggregated  data  are  much  more faSffc  aSsT  K?™h! '? ^  ThS*  *** °f 

£we?Th^ 

it  soVpotentially^sefd  insight         ^      bCC°meS  '°St  ,n  the  aW*&™  Process,  and  with 
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rcertainlv  there  are  many  instances  when  such  aggregated  data  might  be  interpreted 
auali  JveW*  &  STprevtous  discussion.  However,  it  is  recommended  here that  aggregated  data 
£2S S^wheTaffiyzed  statistically,  on  a  sample  large  enough  to  provide  adequate  statistical 
power.) 

7>p«  o/varioWex  When  examining  aggregated  data,  the  investigator  usually  has^ an 
acenda  of  research  questions  which  will  presumably  be  addressed  by  the  date.  OperaUonahang 

SiwSSSwSS.  is  not  a  trivial  matter  because  of  the  wide  range  of  experiences istudente 
tvSLSu rexDerience  during  hypermedia  CBI.  That  is,  the  interpretations  we  might  give  to  certain 
SSSSSSi  S* ffit  might  be  different  from  the  interpretations  we  give  to  anotiier  student 

conclusions  about  what  occurred  during  the  lesson. 

Once  the  specific  variables  are  decided  upon,  the  researcher  needs  to  decide  which  data 
ooints  are  to  be  included  in  the  aggregated  cases  and  variables.  For  example,  say  there  s  an 
SbSotulwh  S^  It  is  a  fairly  simple  matter  to  aggregate  these 

StSSX^to  «S£g  to  the  research  questions  of  interest  The  same  applies  to  any  type 
MwSS>  all  students,  and  generally  fall  into  a  few  common  types :  o ^data: 
nontinal  (e.g.,  multiple  choice,  menu  selections),  counts  or  totals  (e  g.,  numbe  of  times  a  card  is 
Wsited  or  button  is  clicked,  total  elapsed  time  spent  on  a  lesson),  and  averages  (e.g..  average  time 
spent  reading  expository  material  across  a  number  of  such  events). 

However,  there  is  one  type  of  data  researchers  frequently  wish  to  ^Uu^hich i  present 
some  serious  problems  for  analysis.  For  example,  say  the  researcher  wante  to  see  how  many 
tiTes  TSSSSSSSiSric  type  of  help  screen.  Certainly,  the  number  of  times  students  would 
dSSi  to  aSs  he he^wouldte  contingent  upon  how  many  opportunities  they  had  to  access 
he^f  wm^tudents  had  the  opportunity  to  access  help  only  once  or  twice,  how  u i  their  help- 
selection  data  to  be  compared  withsomeone  who  had  many  more  such  help  opportumU^?  Such 
contineentl data  present  continual  analysis  problems  for  researchers  (Misanchuk  &  Schwier,  1992). 
SSSSS^SLc  that  such  proportions  are  ^«r^^bS^l^BI  * 
number  of  possible  transformations  to  better  represent  such  data(Cohen  &  Cohen.  iy  o. 
p.  254-259).  Even  so.  the  interpretability  of  such  data  is  limited. 

More  generally,  in  any  hypermedia  environment,  there  are  likely  to  be  a  great  many  date 
points  which  are  in  some  way  conditioned  on  the  student's  experiences  previously  encountered I  in 
SlS  It   imperative  the  researcher  not  jump  too  quickly  to  aggregate  such  date  witho^ .  first 
carefully  examining  the  range  of  experiences  the  student  traversed  earlier  in  the  lesson.  The  old 
saw  "garbage  in  -  garbage  out"  definitely  applies  here. 

Data  organization  and  format.  Audit  trail  data  generated  by  routines  such  as  the  one 
presented  in  this  paper  do  not  readily  conform  to  the  usual  data  input  requirements  of  statistical 
mSS  H only  the  most  powerful  of  such  programs  (e.g  SPSS)  are  able  to  teke  as  input 
ffitany  kind  of  form  of  data.  Even  so.  such  data  may  still  need  to  ££«J*I^* 
analysis.  Such  mundane  features  as  commas  and  quotation  marks  need  to  be  in  the  proper Tcjmat 
for  input  into  the  statistical  program.  For  example,  the  sample  audit  trail  intos 
paper  would  need  to  be  somewhat  modified  prior  to  statistical  analysis  using  SPSS.  The  string 
tables  and  the  embedded  quotation  marks  would  need  to  be  brought  into  conformity  with  the 
requirements  of  the  program's  data  input  routines. 

It  is  very  likely  that  the  statistical  analyst  will  need  to  employ  other  routines  as  well  (such 
as  SPSS's  INPUT  PROGRAM,  END  INPUT  PROGRAM  routines)  to  rearrange  the  input  data  to 
a  more  acceptable  format 
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Some  analysis  techniques  and  problems.  Of  course,  the  particular  statistical  routines 
employed  will  be  a  function  of  the  research  questions  asked  by  the  investigator.  However,  given 
the  wealth  of  data  collected,  a  number  of  statistical  techniques  (in  addition  to  the  usual  MANOVAs 
and  repeated-measures  analyses)  become  available  and  attractive  due  to  the  existence  for  each 
student  of  a  nch  data  "stream": 

•  ...multi-way  contingency  table  analysis:  related  to  the  problem  of  contingent  data 
mentioned  earlier,  the  investigator  might  wish  to  create  for  each  learner  a  contingency 
table  of,  say,  choices  given  opportunities;  these  tables  can  themselves  are  able  to  be 
analyzed  across  the  entire  sample. 

•  . .  .the  continuous  supply  of  data  from  the  student  over  time  might  offer  researchers 
chances  to  look  for  trend  patterns  over  time.  Techniques  such  as  time  series  and  non- 
linear regression  might  be  useful  statistical  techniques  for  such  analyses. 

•  -  •  occas,onally'  researchers  are  interested  in  classifying  students  by  "type"  or  "style  " 
The  idea  is  to  see  if  students  tend  to  cluster  into  meaningful  groups  bas^d  upon  their 
online pendencies  collected  in  the  audit  trail.  While  fairly  unused  in  educational  circles 
a  number  of  staUsUcal  profiling  and  clustering  routines  exist  to  help  this  sorting  task  ' 
(e.g.,  Hartigan,  1975).  15 

nr  Ty.fisf  Problems  encountered  during  such  analysis  include,  for  example,  choosing  the 

proper  urn  of  analysis  (learner?  Or  where  more  than  one  student  sits  at  a  single  keyboard  should 
die  computer  be  tiie  correct  unit  of  analysis?),  and  determining  whether  a  wi  thin-subjects  design  is 
appropriate  and  desired,  and  if  so,  choosing  the  proper  design  and  error  terms  in  the  model. 

Measurement  Error 

••  u-.£s  wi*  my  data  collection  system,  errors  can  occur  in  the  measures  which  limit  the 
reliability  and  interpretability  of  the  information  gathered.  Although  in  a  computer-based  data 
S  SS^SSl? e  measurement  errors  are  not  of  the  type  we  usually  think  of  when  humans 
collect  and  record  data  (data  entry  mistakes,  lack  of  double  checking,  etc.),  there  are  still  a  number 
of  quality  control  issues  which  must  addressed  in  order  to  have  confidence  m  the^ta.  ^meT 
these  potential  problems  are  of  concern  in  all  measurement  settings;  others  should  be  considered 
mainly  when  preserving  non-aggregated  data  for  qualitative  analysis  purposes;  still  others  are  of 
primary  concern  when  statical  analysis  is  to  be  applied  to  aggregated  audit  teail  data  (In  fact  it 
S™to^  that  what  could  be  labeled  a  sourSof  measurement  erTwten ^SJng  * 
aggregated  data  can  often  be  pinpointed  when  looking  at  the  same  data  in  its  original  non- 
aggregated  form,  that  is,  by  examining  each  student's  audit  trail  individually.) 

General  mea^iY»i^ftntcnnr.<>rrig- 

•  General  disruptions.  Sometimes  students  kick  out  a  power  plug  in  the  middle  of  a  lesson 
or  a  student  receives  a  bad  disk  which  crashes  abruptly,  or  fire  alarms  sound  The 
investigator  must  decide  how  much  incomplete  data  can  be  tolerated,  and  whether  the 
disruption  tainted  the  data  collection  so  as  to  be  invalid  or  otherwise  unusable. 

•  Accidents  and  carelessness.  Students  occasionally  press  the  key  they  did  not  want 
Sometimes  they  realize  their  mistake;  sometimes  they  don't.  How  the  computer  can 
provide  a  means  for  the  studcnt  to  check  and/or  modify  their  answer  without  being  overly 
intrusive  to  the  overall  flow  the  lesson  is  a  common  challenge  to  researchers  * 

•  Hcr<? thc.  invcstIiBator  is  interested  in  what  the  student  is  doing  with  the 

EE fo„^,dvfSrS.S,n8  kcySo     sample,  are  students  talking  with  each  other  or 

SiSlSt  8  SC,rCCnS? ArC  ^tudcn,ts  rcferrin8  to  job-aids,  workbooks,  or  other 
offline  materials?  How  interested  do  thc  students  look?  What  is  the  nature  of  the 
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Sad Sniques  (e.g.,  Webb,  1984)  ormight  be  treated  qual.tauvely. 
^«8iirementmnrftrns  when  analY7'"0  aggregated  daft 

SS^SSXi 'iSLSSStt-^S  individual,  »*m*d  basis. 

Faustina  to  so  through.  The  students  are  in  clanger  of  feelmg  nice  Let  s  just  gei  uin 
SSfr  IKSveness  is  particularly  evident  in  later  stages  of  the  lesson,  as  the 

trails,  perhaps  through  the  "playback"  technique  presented  earlier. 
MsasmgmgJii  cojsgms  when  an?1 v*"g  individual  non-at>gregated  data 

ES^rovSefalse  information  about  the  time-on-task  the  typical  student  might  take. 
S&StoUym  fu?ente  hTve  more  familiarity  with  computers  (keyboards i,  cursors, 

„c^l«~? ranHnm  hut  when  looking  qualitatively  at  individual,  non-aggregatea  data  tne 
SSonTaboutthe  performance  of  a  lesson  from  only  looking  at  a  few  students. 


Summary 

a  method  for  collecting,  "audit  trails"  of  data  generated  by  students  as  they  proceed  through 

computeV&s^  ,^J5SSff SSSt  the 
whSh  completely  capture  the  student's  online  experience,  and  are  able  to  playback  the 

responses  to  the  investigator. 
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n,,r«^Sltionally:  data  ^V8'8  i8Sues  surrounding  the  use  of  audit  trails  were  presented  The 
purposes ofaggregahng  or  not  aggregating  such  data  were  compared,  some  statistician* 

reTie^ed^  mcasurcment  18SUes  P°tential,y  intestability  of  SeoSkL 

In  general,  it  is  expected  that  techniques  and  issues  presented  in  this  paper  will  ao  a  lono 
way  to  improving  our  understanding  of  ho  J  students  experience  computed \Eu£Sm TSid 
how  we  might  improve  our  instructional  designs  to  reflecTthese  humaVfactore^  d 
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Appendix:  Scripts 

All  of  the  handle*  that  follow  arc  tobe  insta^ 
Comments  are  included  in  each  to  make  them  self-documcntmg. 

oaopeaSiack 
StartTroil 
esd  ooenSUck 


oa  SurfTreil 


-  Thia  handler  seta  up  ■  globej  variable  caUsd  Trail. 
It  begiaa  wiU .  kesoer with  bullae,  to Mkate U* 

-  stsrt  of  a  sew  play  of  the  auct  followed  by  ■  line 

-  skewing  tka  at^uat  time  and  dun. 

-  IV  third  line  in  left  Wank,  but  stacks  cm  V  nenpted 

-  to  put  the  user's  mm,  ixperimisUl  condition,  or  other 

-  dau  ktra. 

tlooalaunTlrne,Uill,tvtat,UalKva»rt,ima 

put  w  A  return  isU)  (rail 

but  'Slack  opened  aiB  e\A  tka  long  lime  afUr  trail 

put  •  on'  eYeVtbe  toni dau  *  rttum efter trail 

•ctaumUrFormatto'O.OO1 

put  tka  ticks/60  Into  startTim* 

put  Ointo  avast 

put  nUitTlme  Into  InktbvantTlms 
snd  SUrtTrait 


on  close  Stack 
SevaTrailFile 
end  closeSuck 


oa  SaveTrailFile  .  .  . 

Tkia  writes  tke  Trail  global  variable  to  dink 


.."  End  the  trail  global  with  closing  time  and  elapsed  time 

global  sUrtTime.trail 

put  tke  ticks/60  into  endTime 

act  oumberFormat  to  "O.CO"  .  ft  , 

r*turti  Jtr  •Suck  cloned  at*  &&  the  one  time  &  return  after  trail 
Tut  TSttaH  fcTSfik ^dTTme -«*rn«ime  AA  •seconds*  A  return  after  trail 
put  'QQQQQQ*  *  return  after  trail 


mm  The  next  line  specifies  the  volume  sad  name  of  the  trail  file. 

By  default,  TrailFile*  wUl  V  created  in  the  same  folder 
-as  HyperCard  itself.  If  you  wanted  to  put  it  awwwnere  else 
..  you  would  put  the  full  pathname  into  the  varuWe  TrailFile. 
~  i^exajn^Tput  •CirrusSys:DaU:™ 
~  would  define  TrailFile  aa  a  Tile  called  TmlFde 
mm  in  a  folder  called  Data'  on  a  drive  named  OmisSys 

put  TrailFile*  into  TrailFile 

Ne^  read  to  the  end  of  the  trail  file,  if  it  exists,  so 

-  that  this  latest  trail  will  be  appended  to  it  If  there  la 

-  no  trail  file,  this  line  creates  one. 

open  file  TrailFile 
repeat 

read  from  file  TrailFile  for  16384 
if  it  is  empty  thai  exit  repeat 
end  repast 

write  trail  to  file  TrailFile 
close  fiU  TrailFile 
end  SaveTrailFile 
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<*  AddToTrail  imp* 

-  This  headier  Mora  data  about  moueeclicks  is  a  iliac  "ia  a" 

-  global  called  Trail  aloag  with  the  time  at  which  they  occurred 

-  aid  the  target  of  the  moaaechck. 

-  Ilem  1 « the  eveat  aember,  etartiag  with  1 

--  Item  2  m  the  total  elapsed  time  la  secoade  etace  the  stack  ooeead 
~  tem  3  ■  the  tars*  cf  the  moeaedick  (a  field,  buttoa  or  card) 

-  Item  4*  the  aame  of  the  card 

-  Item  5  m  the  elapaed  time  aiaoa  the  laat  mouaeclick 

-  Item6Jt7«taacootdiaatc*of  ttemoaaeclick 

-  Item  8  Jfc  9  » the  coordiaatca  whew  taa  mouse  buttoa  waa  mlaaaed 

-  Item  10  *  Noa-click  aveate  each  aa  pulliag  dowa  meaua  or 

aaer  iapat  to  a  field.  This  is  a  parameter  paaaad 

whea  aeceaaaiy.  See  the  domeau  headier,  for  example. 

global  sta!fTime,treiI,eveat,laa^ 

pat  the  tkka/60  iato  aowTime 

if  DoatAddToTrail  -  trae  tSea  exit  AddToTrail 

add  1  to  eveat 

pat  aveate  3  iato  traJIUae 

pat  traac(svset)  iato  hem  1  of  liae  trailUae  of  trail 

•at  aumherPoraaal  to  "0.00" 

pet  aowTime  *  starfTime  iato  item  2  of  liae  trailUae  of  trail 
fUl  Ik?  °^  ^*  card  iato  ilem  3  of  Uae  trailUae  of  trail 

put  the  asms  of  the  target  iato  ham  4  of  liae  trailUae  of  trail 
**%  IS?*]?1?? "  b*E»"*Timt  iato  item  5  of  liae  trailUae  cf  traU 
put  the  clkkloc  iatohem  6  to  7  of  liae  trailUae  of  trail 
put  the  moaaaloc  iato  item  8  to  9  of  liae  trailUae  of  trail 
put  iapat  iato  kern  10  of  liae  trailUae  of  trail 
put  aowTime  iato  laetEveatTime 
aad  AddToTmU 


oa  domeau  x 

AddToTrail  "Ma  a  a  hem"  A  at  x 
pam  domeau 
cad  domeau 


ofiShoa/frail 


-  Thia  headier  createa  a  represeatatioa  of  the  aaage  pattens 

-  of  a  stack.  It  pence  a  trail  file  coaUiaiag  date  from 
■■  oae  or  more  niaa  of  a  atack  aad  marks  the  locatioa  of 

-  every  moaaeclick  oa  each  card. 

-  To  execute  thia  headier,  call  up  the  message  box. 
~  Sje  'SaowTraa",  aad  prces  the  return  key. 

-  Ttoe  should  be  doae  oa  a  COPY  of  the  stack. 

-  The  first  eectioe  creates  a  aew  buttoa  earned  Breadcrumb 

-  la  the  form  of  a  email  shadowed  circle. 

global  DoatAddToTrail 

put  true  iato  DoatAddToTrail 

domeau  aew  buttoa 

get  the  id  U cd  bta  "New  Buttoa" 

put  it  iato  x 

set  the  ahowMameof  cd  bta  id  x  to  false 
set  the  aame  of  cd  bta  id  x  to  "Breadcrumb" 
set  the  rect  of  cdbtaid  xto  100,100,111112 
•elect  cd  bta  id  x 
domeaa  "Cat  Buttoa" 


"■  Next  we  locate  the  file  that  captured  the  trail 


aacwer  file  "Whew  ie  the  trail  file?" 
if  it  ia  aot  empty  thea 
out  it  iatoTraflBle 

cbooec  browse  tool 
put  falae  iato  DoatAddToTrail 
exit  ShowTrail 
ead  if 

opeafileTrailRle 
put  V  iatoTheUae 
•et  tockjaaaaagea  to  true 
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I  'NexCiw  ^  tiira  each  lb*  of  the  TrailRle.  lookieg 
-for  lines  rcpresentisg  inoiiseclicks 


repeat  until  thelJne  it  empty 
read  from  file  TmURk  until  return 
puts  into  theiine 

ITkVoMofuie  lines  with  date,  "•",  etc..  skip  it 
if  tJ^  lumber  of  iumc  in  theline  <  9  then  next  repeal 


-  Oo  to  the  card  where  the  event  occurred 


put  item  3  of  theline  into  theCud 
&word2oftheCwd«"id"  tnea 
do  "go"  kJk.  theCard 
etoe 

do  "to  card*  k&  quote  A  tneCard  ft  quote 
end  if 


-1  Find  the  location  of  the  monaeclick,  ignoring  pull 
~  menus  and  inputs  into  a  field,  and  paste  the  Bit* 


put  item  6  to  7  of  theline  intoOjeLocation 

put  item  10  of  theline  iato  nonClkk 

if  nonOick  contains  "Menu  item:"  or  nooOjck 

next  repeat 
else 

domenn  -Paste  Button" 

put  the  number  of  cd  btne  into  x 

tetthelocofcdbuxto  theLocaUon 

end  if 
end  repeat 


«  Now  tidy  things  up 


close  fileTmilRle 
choose  browse  tool 
put  false  into  DontAdoToTratl 
ptay  harpsichord 
end  showrrail 
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Constructivism  and  Instructional  Design: 
Some  Personal  Reflections 

Brent  G.  Wilson 
University  of  Colorado  at  Denver 

The  field  of  instructional  design  (ID)  is  in  a  state  of  rapid  change.  Recent 
expressions  of  constructivist  theorists  (Bednar,  Cunningham,  Duffy,  &  Perry 
1991)  have  engendered  a  lively  debate,  reflected  in  the  May  and  September  1991 
issues  of  Educational  Technology  magazine  and  the  winter  1992  issue  of  ETR&D 
If  our  IT  department  is  any  indication,  graduate  students  across  the  nation  are 
engaging  their  professors  in  heated  discussions  concerning  the  fundamental 
models  in  our  field,  and  how  they  hold  up  to  the  constructivist  onslaught 

This  is  good.  I  feel  better  about  the  future  of  ID  than  I  have  since  I  was  a  gradu- 
ate student  myself  For  years,  Dave  Merrill  has  pled  that  more  serious  attention  be 
given  to  ID  theory  development.  He  finally  seems  to  be  getting  his  wish,  but  perhaps 
not  the  result  he  anticipated.  For  a  long  time,  thoughts  about  the  nature  of  ID  theory 
and  its  practice  have  been  fermenting  in  my  mind;  this  paper  is  my  forum  for 
airing  some  of  those  thoughts  and  sharing  them  with  a  wider  audience.  The  tone  is 
personal  because  it  deals  with  underlying  assumptions  I  have  made  in  doing  ID 
In  a  way,  the  paper  is  a  sort  of  confessional-my  tone  and  stance  are  much  less 
temperate  than  any  of  my  previous  writing.  Narrative  forms  of  research  are 
ESS  n  eaimne  esteem  among  educators  and  social  scientists  (Polkinghorne, 
1988  Connelly  &  Clandmin,  1990;  Witherell  &  Noddings,  1991);  I  ask  you  to 
consider  this  paper  a  sort  of  narrative  documentation  of  my  professional  beliefs 


Th  4  Problem:  ID  Is  Out  of  Sync 

Ae  problem  can  be  simply  stated:  ID,  in  its  present  form,  is  out  of  sync  with  the 
times. 

•  Its  orientation  is  behavioristic. 

•  Its  methods  are  behavioristic. 

•  Its  research  base  is  behavioristic. 

I  know  that  these  claims  make  some  people  mad,  particularly  people  who  believe 
«A^%CTP  Cted  thC  "c°enitive  revolution.-  They  are  generalizations,  and 
admittedly  there  are  exceptions.  Perhaps  a  better  descriptor  would  be  'positivistic' 
rather  than  «behavionstic.'  But  the  image  of  ID  to  outsiders  is  definitely  behavior- 
istic. Survey  any  group  of  faculty  within  a  college  of  ed;  you  will  find  a  stereotyped 
impression  of  ID  as  no-nonsense,  results-driven  behaviorism  applied  to  education. 

Mmilar  image  holds  among  educational  researchers:  members  of  the  IT  SIG 
group  within  AERA  tend  to  be  seen  as  positivists  par  excellence 

To  most  of  us  this  comes  as  no  surprise.  To  some,  our  positivistic/  behavioristic 
™,t/l  •  pr°b>m\.In  the  current  Zeitgeist  of  increasing  constructivism,  how- 
ever, behavioristic  notions  of  ID  find  themselves  increasingly  out  of  the  main- 
stream of  educational  thought.  I  wonder,  prior  to  the  Ed  Tech  issues,  how  many  ID 
S^S?    ftv  I  confronted  the  disparity  between  progressive  educational 

thought  and  the  ID  models  learned  in  classes.  To  many,  the  Ed  Tech  issues  must 
have  contributed  to  a  loss  of  innocence.  The  line  is  now  clearly  drawn,  and  I  sus- 
pect many  find  themselves  uncomfortable  as  they  see  unwanted  conceptual  bae- 
gage  accompanying  the  tools  and  models  of  their  field. 

It  is  possible,  though,  that  I  am  merely  projecting  my  own  past  perceptions  and 
experiences  onto  others.  As  a  graduate  student,  I  felt  privileged  to  have  worked 
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&SeT  1980X     bought  into  the  field-its  theories,  models,  and  aim*-yet A 

Sit  ^^^'ffSSffiS 

F     ,  F.,       j  <i;ffir»itv  of  desim  decisions  always  made  me  nesitani,  auuut 

tt.se  X  Mto°that  instruction  can  be  deigned  to  te»h  knowledge  and  those 
wTbetve  that  knowledge  i.  constructed  by  learner.  I  would  g^ m»  ferto  -ay 
That  the  two  positions  are  irreooncilable.  If  you  accept 

rists"  sav  then  you  cannot  simply  revise  [elaboration  theory]  in  the  ways  you 
Pr£.se :  toXke  it  fit  in  with  these  view,  You  have  to  throw  it  out £*M 
Tm  not  a.  pessimistio  about  the  possibility  of  reconchation.  ft /bot,  thai » 

t  oy  nofto  Way  problems,  but  to  be  unflinchingly  honest  and  straightfor- 
wa  ayin  criUcfsm,  U  aggressive  in  offering  alternative Resign  c-gfi  g 
Mnerience  has  now  started  me  wondering  how  many  other  professionals  wno, 
rafher  Sian con front  their  problems  with  the  models,  have  simply  walked  away 
from  .he  problem  by  disengaging  themselves  from  the  models  or  from  the  field. 

^X^Xf^^  obiectivist/constructivist  dialogue. 
Rattier  ftan  compr^ensively  examine  the  issue,  I  want  to  raise  a  few  pointe 
V  You  dolt  have  to  be  a  philo,opher  to  take  a  po,iHon.  Among  the  to  Tech 
conSSutors,  Perkins  and  Cunningham  have  considerable  W^M; 
but  to  my  knowledge  the  remaining  writers  are,  like  you  and  me,  relative  nonces 
L  that  domain   Our  interest  in  these  issues  derives  from  a  desire  to  do  the  right 
Sing  to  bTknowledgeable  about  sources  of  bias  and  *«^«™*™  ™  £ 
ma£  in  our  work    I  hope  that  practitioners  and  researchers  do  not  defer  to  tne 
"xp  As-  on  issues  so  central  t.  defining  the  field   At  the  same  time  I  am  busy 
trX  to  expand  my  breadth  and  knowledge  base  about  philosophy  As  I  read  more 
Sn,  moTBruner,  and  more  postmodern  philosophy  my  pe ^eeSng 
necessarily  influenced.  That  change  in  perception  ^^^^SmlSSi 
that  I  am  growing  in  understanding  the  meaning  of  my  field.  (And  L  «™"»  1 
to  growTmpatient  with  fellow  researchers  who  show  no  inclination  to  broaden  their 
understanding  of  the  issues  squarely  facing  the  field.) 
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How  do  our  ideas  of  structure  relate  to  learning/  memory  theorists'  ideas  about 
how  memory  is  organized?  Mayer,  Greeno,  Ausubel,  Kintsch,  Craik  &  Tulvine, 
Norman  &  Lindsey,  Quillian  &  Collins,  McConkie,  etc.... 

The  problem  of  "segmentation":  Are  we  sure  the  contents  of  a  discipline  can  be 
broken  down  into  individual  concepts,  principles,  etc.?  What  about  what  theorists 
call  verbal  information,"  "meaningful  verbal  materials"— facts  that  are  im- 
portant to  know  but  don't  connect  to  any  principles  directly....Are  we  rather  talking 
about  a  mere  skeleton  in  our  work,  which,  to  be  complete,  needs  to  be  fleshed  out  by 
facts,  details,  context,  etc.?.... 

It  seems  to  me  that  there  are  certain  kinds  of  content  whose  purposes  do  not  admit 
to  use  of  synthesizers/epitomes  on  an  elaborative  approach.  Narratives,  for 
example,  often  have  a  dramatic  component,  as  if  the  teacher/author  were  telling  a 
ST?',.   -  °uld  1)6  senseless  to  le*  *e  cat  out  of  the  bag  prematurely  simply  to 
stabilize  the  content  for  the  learner.  An  effective  alternative  (in  certain 
situations)  is  having  the  student  on  the  edge  of  his  seat  waiting  in  suspense  for  what 
happens  next.... 

Bruner  (1966)  distinguishes  between  passive  and  active  learning,  between  what 
we  know  and  what  we  do  with  what  we  know.  How  do  these  seemingly  separable 
kinds  of  knowledge  relate  to  our  scheme?  Is  true  active  learning  on  the  rule- 
finding  or  rule-using  level?  Are  procedures  and  principles  real  problem-solving 
behaviors?  Would  Ausubel  object  that  we  devalue  facts  and  verbal  information 
unnecessarily?.... 

Is  'content'  defined  as  "What  is,"  "What  is  presented  to  the  student,"  or  "What  is 
expected  to  be  learned?"  

Table  1.  Excerpts  from  my  notes  on  elaboration  theory,  circa  1978. 


2.  Nobody  I  know  admits  to  being  an  objectivist.  Molenda  (1991)  points  out  that 
objectivism  is  pnmarily  a  pejorative  label  given  by  constructivists  to  the  offending 

aT;  ^ak°ff'  19?7;  Bmner'  1986)-  Th*  fact  alone  »  causf 
to  worry.  It  s  hard  to  talk  seriously  about  a  philosophical  position  that  no  one  admits 

to.  This  goes  for  caricatures  made  by  both  ideological  sides  of  the  Ed  Tech  dia- 
logues. Very  few  people  hold  radical  positions  of  either  persuasion 

Please  note  that  my  complaint  has  to  do  with  labels  and  descriptions;  many  peo- 
ple may  not  call  themselves  objectivists,  but  their  way  of  seeing  may  be  very  differ- 

SL^!  C°n??UC!?lBt  perfpective;  Philosophers  holding  this  more  traditional 
view  of  the  world  call  themselves  realists  (House,  1991;  H.  Putnum,  1990).  To  my 
understanding,  realists  do  believe  there  is  a  "reality"  out  there,  and  that  the  quality 
of  mental  representations  can  be  judged  by  their  correspondence  to  that  reality 

Ohio XP °i?ienCe  thTy  °f  trLUth"  is  one  of  ma°y  issues  hotly  debated  by 
philosophers  If  they  can  disagree  about  it,  I  imagine  the  rest  of  us  can  as  well. 
The  cross-talking  going  back  and  forth  about  constructivism  reminds  me  of  an 

2£u A  I™6"  y.„.ead  m  ReaderS  Digest  (Safire-  1991>  about  a  Florida  politician, 
asked  to  take  a  position  on  a  county  option  to  permit  the  sale  of  liquor 

If ^by  whiskey,  you  mean  the  water  of  life  that  cheers  men's  souls,  that  smooths 

out  the  tensions  of  the  day,  that  gives  gentle  perspective  to  one's  view  of  life,  then 

put  my  name  on  the  list  of  the  fervent  wets. 

"But  if  by  whiskey  you  mean  the  devil's  bew  that  rends  families,  destroys  careers 
and  ruin  s  ones  ability  to  work,  then  count  me  in  the  ranks  of  the  dries."  (p  14) 

M-in  C0MSt,rUCiiviim  y°U  mean  the  ^llP"8"1        subjectivism  portrayed  by 
'If  Molenda  then  I  am  a  strident  opponent.  But  if  by  constructivism  yol 
rnnnVL  ■   reasoned' P6™"^  philosophy  of  a  Cunningham  or  a  Perkins,  then 
count  me  in  as  a  constructivist. 
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ArftL , opting  «h.  IneU&er  of  the  box,  with  the  ».>m» ,«* ,ec »W  * 
from  the  nonhuman,  objective  world,  Heidegger's  analysis  leads  to  the  conelu 
•Ton*.'  taZ.  being  is  already  being-in-the-world.  There  is  no  .na.de  walled 
off  from  the  outside.  (Faulconer  &  Williams,  p.  46)  .,QT4;na 
AccorS  To  Heideggar  and  other  P^en^ 

point  is  recognizing  that  we  simply  are  in  the  world,  working,  °0,nj 
SStogs   Turntag  Descartes'  famous  maxim  on  its  head,  the  motto  becomes  /  am 

^^^^^on  is  dethroned  as  the  center  of  the  universe  and 
pwJacS  the  Context  ofbeing  part  of  the  world.  This  philc ^^JX 
Sent  of  the  socially  oriented,  connected  ways  of  knowing  ^J^^9?* 
Belenckv  and  colleagues  (Belencky,  Clincy,  Goldberger,  &  Tarule,  1986).  I  am 
attracted  to  such  holSic  conceptions  of  the  world,  even  if  my  understanding of the 
phTbsophy  is  still  incomplete  (see  Polkinghorne,  1990  for  a  good,  short 
to  many  of  the  issues).  My  reading  is  enough  to  make  me  suspect  that  much  of  the 
objective/constructive  debate  is  based  on  the  wrong  question. 

4.  Constructive  is  a  philosophy  not  a  strategy.  I  ^^^^SSZ- 
lar  instructional  strategy  is  inherently  constructive  or  objectivistic  Construct 
Msm^ofan  instructional  strategy  to  be  deployed  under  appropriate  cond^ons. 
Rather,  constructivism  is  an  underlying  philosophy  or  way  of  seeing  the  world. 
This  way  of  seeing  the  world  includes  notions  about:  ,    .    ,  _f _fll. 

•  the  nature  of  reality  (mental  representations  have  "real"  ontological  status 

.  C^Tl^f^t*^  constructed;  it  is  inside  people's 

.  Mature  oThutZlteraction  (we  rely  on  shared  or  "negotiated"  meanings, 

better  thought  of  as  cooperative  than  authoritative  or  manipulative  in  nature) 
.  Ae  naturegof Science  (it  is  a  meaning-making  activity  with  the  biases  and 
filters  accompanying  any  human  activity)  different 
When  we  see  the  world  in  constructivist  terms,  we  go  about  our  jobs  in  a  different 
way,  but  the  difference  cannot  be  reduced  to  a  discrete  set  of  rules  or  techniques.  Let 

^WoTrmecPently  turned  eight,  and  for  his  birthday  we presented him  with > 
computer  math  drill-game.  He  is  a  good  math  problem- solver-he  likes  to  play 
S3 1  numbers  and  invent  routines  -but  because  of  a  schooling  mixup,  he  is  behind 
Tn  mastering  his  math  facts.  I  essentially  said,  "Joel,  this  is  a  fun  game  you  will 
enjoy  it  will  also  help  you  learn  the  addition  and  subtraction  skills  that  youre  a 
Stto  behind  in.  It's  no  big  deal.  You  will  be  glad,  became  learning  that  stuff  is 

80  Do^my^rofa'drm-and-practice  program  make  me  an  objectivist?  I  deny 
that  it  do^s  I  also  deny  that  I  am  violating  my  deeply  held  constructivist  principles 
abou  people  and  the  way  we  learn.  People  do  construct  meaning  from ,  tt« -mg» 
ences  learning  should  be  meaningful  and  derive  from  an  authentic  context, 
peo  e  shouS  be  allowed  to  pursue  individuallearning  goals.  I  believetha  Joel  has 
a  pretty  good  idea  of  what  the  game  is  doing  for  him,  and  the  kind  of Tun  he is< ieny 
ine  from  it  As  he  chooses  to  make  use  of  the  game,  he  is  actively  constructing 
meaning  and  new  knowledge.  Joel  has  plenty  of  other  opportunities  to  exercise  his 
more  creative  talents;  his  use  of  the  game  is  filling  a  needed  learning  gap  to  meet 
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the  expectations  and  pace  set  by  his  school.  He  was  much  happier  in  school  two 
weeks  later,  when  he  aced  the  timed  math  test  and  came  home  to  tell  us  about  it. 

My  point  is  that  a  given  instructional  strategy  takes  on  meaning  as  it  is  used,  in 
a  particular  context.  If  I  had  tricked  my  son,  "Joel,  look  at  this  computer  game.  It's 
better  than  Nintendo!"  and  pretended  that  he  was  already  great  at  his  math  facts, 
and  that  the  game  had  no  bearing  on  his  schooling,  then  I  would  have  felt  in  viola- 
tion of  my  philosophy.  So  the  same  instructional  technique  could  have  vastly 
different  meaning  (and  effects)  depending  on  its  context  of  use. 

Another  example— a  journal  entry  from  Scott,  a  teacher  in  my  Reflective 
Educator  class  last  semester: 

Third  hour  composition  I  went  to  a  seating  chart,  the  first  time  I've  done  that  here. 

I  caught  them  as  they  came  in  and  told  them  where  to  sit.  Great  improvement 

Everyone  working  hard  on  their  papers....I  sense  the  students  are  relieved  that 

I  ve  imposed  more  structure. 
Scott  teaches  at  an  alternative  high  school.  His  philosophy,  as  expressed  in  journal 
entries,  class  contributions,  and  teaching  methods,  is  definitely  constructivistic 
and  anti-authoritarian.  Yet  imposing  a  seating  chart  on  a  class  is  a  clear  act  of 
asserting  authoritative  control  and  imposing  structure.  Is  Scott  betraying  his  prin- 
ciples, or  can  an  ostensibly  "constructivist"  instructional  technique  actually  serve 
his  constructivist  learning  and  teaching  goals?  The  students*  answer  to  that  is 
clear:  they  welcome  the  new  arrangement  and  view  it  as  supporting  their  own 
learning  goals. 

Too  often,  constructivism  is  equated  with  low  structure  and  permissiveness— 
imposing  predefined  learning  goals  or  a  learning  method  is  somehow  interfering 
with  students  construction  of  meaning.  In  extreme  cases,  that  may  be  true.  Yet  to 
help  students  become  creative,  some  kind  of  discipline  and  structure  must  be 
provided.  Laurel  (1991)  cites  Rollo  May  (1975),  who  makes  this  point  very  well- 
Creativity  arises  out  of  the  tension  between  spontaneity  and  limitations,  the  latter 
(like  river  banks)  forcing  the  spontaneity  into  the  various  forms  which  are 
essential  to  the  work  of  art....The  significance  of  limits  in  art  is  seen  most 
clearly  when  we  consider  the  question  of  form.   Form  provides  the  essential 
boundaries  and  structure  for  the  creative  act.  (quoted  in  Laurel,  1991,  p  101) 
In  other  words,  an  instructional  strategy  that  imposes  structure  may  actually  help 
learners  make  constructions  needed  for  learning.  Joel's  computer  game  or  Scott's 
seating  chart  may  be  hindering  or  serving  constructivist  learning  goals:  You  can't 
tell  by  looking  only  at  the  strategy;  you  have  to  look  at  the  entire  situation  and  make 
a  judgment.  That  is  the  role  of  the  teacher  or  instructional  designer:  to  make 
professional  judgments  about  such  things. 

Implications  for  Design  Theories1 

In  this  section,  I  turn  to  the  more  difficult  issue  of  defining  ID  theories  and  their 
relation  to  practice. 


iXlIES.Kf  V  XoJie  u?e  01  ,he  ,erm  ',heory  when  applied  to  instructional  design 
SSSSMJWJS^.  W  n°,hin°  m0re  ^ te<*™'°0ical  moSels.  I  retain 
.H    .1 }  us8  of  1neory  ,or  two  rcasons:  (1)  to  distinguish  between  ID  theories 
SSSKFirt  dev^Prnenl  mode's  that  prescribe  a  methodology  for  managing  ID 
?$S£\A??TS  &  £°odson- 1980).  and  (2)  because  philosophers  of  science 

ZSSlSnStX"*  !h e0fy' a^ ,model" 10  a  varie,y  01  '^ewofts  of  varying 
complexity,  formality,  and  power  (Suppe,  1977). 
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What  is  Instructional-Design  Theory? 

Traditionally,  ID  theories  are  seen  as  prescriptive  in  the  the  sense  that 

•  they  provide  recipes  or  heuristics  for  doing  designs,  and 

•  they  also  specify  how  end-product  instruction  should  look. 

Thus  in  both  a  product  and  process  sense,  ID  theories  serve  as  guides  to  professional 
practice.  Conceptually,  ID  theories  are  much  closer  to  engineering  than  to  science. 
They  are  about  how  to  get  something  done,  how  to  design  a  solution  not  about  how 
the  world  is.  In  that  sense,  they  are  really  less  theories  and  more  models  for  action, 

for  problem  solving.   

Such  design  theories  may  be  based  on  a  lot  of  hot  air,  or  they  may  have  some 
validity.  What  kind  of  knowledge  base  are  these  theories  built  on?  Several  forms 
of  knowledge  may  contribute  to  an  ID  model,  including: 

•  scientific  knowledge  of  learning  and  related  sciences 

•  craft  knowledge  of  effective  design,  based  more  on  teaching  practices  than  on 
formal  scientific  research  ,    T_      .  . 

•  idiosyncratic  knowledge  about  instruction  unique  to  the  ID  protession, 
untested  by  formal  research  yet  functionally  important  to  ID  practice 

Reigeluth  (1983a)  has  outlined  a  prescriptive  framework  for  embodying  this 
knowledge.  A  series  of  rules  are  developed  connecting  existing  conditions, 
desired  outcomes,  and  recommended  methods  to  instrumentally  obtain  those  out- 
comes. For  example,  if  your  learners  are  new  to  a  concept  and  you  want  them  to 
learn  it  at  an  application  level,  then  you  might  present  a  statement  of  the  definition 
followed  by  examples  and  practice  opportunities  to  classify  new  cases.  An  ID 
theory  builds  a  collection  of  similar  IF-THEN  rules;  designers  are  then  supposed  to 
apply  those  rules  to  their  various  situations. 

This  is  a  fairly  technical  view  of  design  activity.  Schon  (1987),  in  fact,  refers  to 
exactly  this  type  of  thinking  as  technical  rationality. 
From  the  perspective  of  technical  rationality...a  competent  practitioner  is  always 
concerned  with  instrumental  problems.  She  searches  for  the  means  best  suited  to 
the  achievement  of  fixed,  unambiguous  ends... and  her  effectiveness  is  measured 
by  her  success  in  finding...the  actions  that  produce  the  intended  effects  consistent 
with  her  objectives.  In  this  view,  professional  competence  consists  in  the  applica- 
tion of  theories  and  techniques  derived  from  systematic,  preferably  scientific 
research  to  the  solution  of  the  instrumental  problems  of  practice,  (p.  33) 
Schon's  technical  rationality  looks  a  lot  like  Reigeluth's  conditions-outcomes- 
methods  framework.  Schon  does  not  deny  that  some  problems  encountered  are 
routine  ones  that  relate  to  the  rules  and  concepts  of  the  discipline.  However  profes- 
sionals go  far  beyond  technical  rationality  when  they  encounter  novel  problems: 
There  are  also  unfamiliar  situations  where  the  problem  is  not  initially  clear  and 
there  is  no  obvious  fit  between  the  characteristics  of  the  situation  and  the  available 
body  of  theories  and  techniques.  It  is  common,  in  these  types  of  situations,  to 
speak  of  "thinking  like  a  doctor"— or  lawyer  or  manager  tor  instructional 
designer!]....  ,        .  ... 

We  would  recognize  as  a  limiting  case  the  situations  in  which  it  is  possible  to 
make  a  routine  application  of  existing  rules  and  procedures.... Beyond  these 
situations,  familiar  rules,  theories,  and  techniques  are  put  to  work  in  concrete 
instances  through  the  intermediary  of  an  art  that  consists  in  a  limited  form  of 
reflection-in-action.  And  beyond  these,  we  would  recognize  cases  of  problematic 
diagnosis  in  which  practitioners  not  only  follow  rules  of  inquiry  but  also  some- 
times respond  to  surprising  findings  by  inventing  new  rules,  on  the  spot.  (Schfcn, 
1987,  p.  35) 

Technical  rationality  suggests  a  clear  demarcation  between  theory  and  practice, 
with  categories  of  basic  knowledge,  applied  knowledge,  and  practice.  Theory  is 
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what  gets  written  in  textbooks  and  professional  journals,  while  practice  tends  to  be 
mistrusted  since  practitioners  never  have  the  good  sense  to  apply  theory  correctly. 
On  the  other  hand,  SchSn's  reflective  practitioner  model  blurs  the  line  between 
theory  and  research,  suggesting  that  practitioners  embody  personal  theories  of 
practice,  and  often  assume  a  kind  of  research  stance  toward  their  work  (see  also 
Winn,  1990).  Sch6n  makes  clear  the  philosophical  basis  of  his  view  of  practice: 
Underlying  this  view  of  the  practitioner's  reflect-in-action  is  a  constructionist 
view  of  the  reality  with  which  the  practitioner  deals— a  view  that  leads  us  to  see  the 
practitioner  as  constructing  situations  of  his  practice,  not  only  in  the  exercise  of 
professional  artistry  but  also  in  all  other  modes  of  professional  compe- 
tence....[0]ur  perceptions,  appreciations,  and  beliefs  are  rooted  in  worlds  of  our 
own  making  that  we  come  to  accept  as  reality.  (Schon.  1987,  p.  36) 
In  contrast, 

technical  rationality  rests  on  an  objectivist  view  of  the  relation  of  the  knowing 
practitioner  to  the  reality  he  knows.  On  this  view,  facts  are  what  they  are,  and  the 
,    truth  of  beliefs  is  strictly  testable  by  reference  to  them.    All  meaningful 
disagreements  are  resolvable,  at  least  in  principle,  by  reference  to  the  facts.  And 
professional  knowledge  rests  on  a  foundation  of  facts,  (p.  36) 
Thus  Sch6n  is  setting  certain  conditions  for  a  constructivist  model  of  instruc- 
tional design.  If  you  buy  into  a  constructivist  idea  of  the  world,  learners  become 
active  creators  of  meaning  (and  teachers  to  others),  and  teachers  are  continual 
learners. 

Communities  of  practitioners  are  continually  engaged  in  what  Nelson  Goodman 
(1978)  calls  "worldmaking."  Through  countless  acts  of  attention  and  inatten- 
tion, naming,  sensemaking,  boundary  setting,  and  control,  they  make  and 
maintain  the  worlds  matched  to  their  professional  knowledge  and  know- 
how.. ..When  practitioners  respond  to  the  indeterminate  zones  of  practice  by  hold- 
ing a  reflective  conversation  with  the  materials  of  their  situations,  they  remake  a 
part  of  their  practice  world  and  thereby  reveal  the  usually  tacit  processes  of 
worldmaking  that  underlie  all  of  their  practice.  (Schtfn,  1987,  p.  36) 
I  have  cited  Schtfn  heavily  because  I  am  convinced  that  we  need  to  rethink  the 
roles  of  formal  ID  theory  and  the  ID  practitioner  (see  also  Hoshmand  &  Polking- 
horne,  1992).  SchSn's  model  is  not  complete— for  example,  he  does  not  pursue  the 
ethical/moral  dimensions  of  professional  decisionmaking  in  an  institutional 
context.  But  his  views  serve  as  a  valuable  starting  point  for  discussion. 

The  expert/novice  literature  within  cognitive  psychology  reaches  similar  con- 
clusions about  the  nature  of  expertise.  Researchers  have  found  that  expertise  is 

•  largely  intuitive  and  inaccessible  to  direct  reflection  (e.g.,  Bloom,  1986) 

•  more  pattern-matching  than  rule-following  (Suchman,  1987,  Bereiter,  1991) 

•  more  qualitative  than  quantitative  (White  &  Frederiksen,  1986) 

•  highly  context-  and  domain-dependent  (Brandt,  1991). 

Such  a  view  of  expertise  seems  also  to  fit  the  field  of  ID.  We  know  that  profes- 
sional designers  are  highly  flexible  and  adaptive  in  applying  their  knowledge  to 
working  problems  (Nelson  &  Orey,  1991;  Thiagarajan,  1976;  Nelson,  Magliaro,  & 
Sherman,  1988);  moreover,  their  use  of  knowledge  seems  to  differ  significantly 
from  formal  theories  (Tessmer  &  Wedman,  1992;  Wedman  &  Tessmer,  1990). 
With  this  view  of  expertise,  the  precise  role  of  traditional  theory  is  left  in  question. 
If  ID  theories  are  not  descriptive  science,  and  not  a  set  of  rules  to  be  unambiguously 
applied  to  problems,  then  what  are  they  and  what  value  do  they  have? 

One  possibility  is  that  the  recipes  contained  in  ID  theories  may  have  some  value 
to  novice  designers.  R.  Putnum  (1991)  found  that  when  learning  a  complex  subject, 

rices  tend  to  grasp  onto  formulas  and  recipes  to  support  initial  performance,  then 
chang  their  use  of  the  recipes  as  they  gained  expertise.  The  main  problem  with  this 
rationale,  though,  is  that  ID  theories  are  not  formulated  as  simple  recipes;  they  are 
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not  easy-to-use  "hooks"  into  a  subject.  Rather,  ID  theories  are  typically  "Presented 
af  Sal  technical-sounding  systems  with  extensive 

acronyms  The  ID  theory  papers  I  have  read  are  anything  but  a  support  to  novices 
?D Tories  are  written  as  though  they  were  serious  science;  novices  require  another 
type  of  representation  altogether  to  support  their  initial  learning  needs. 

Another  possibility  is  that  ID  theories  are  not  really  meant  to  be  used  by  human 
deefe r informal  situations,  but  rather  are  best  suited  to  computer-based  tmn- 
ingfnd "automated  instructional  design.  This  possibility  is  much  more  promis- 
ing, even  though  both  Gagne  and  Merrill  have  denied  ^.^^^SuSS 
lations  of  ID  theory  for  computers  and  traditional  media  (Gagne,  1988,  Merrill, 
1988)    Of  Zse,  tfiere  is  still  the  question  of  whether 

represent  design  knowledge  with  computers.  It  may  turn  out Jtoattiie  Il^THEN rule 
approach  to  design  may  be  unable  to  capture  true  design  expertise.  This  is,  how 
ever  an  empirical  question  worthy  of  continued  investigation. 

In  summa  y'm  theory,  with  Us  prescriptive  orientation  toward  ^th  Procedure 
and  P  oduct,  L  in  conceptual  limbo.  Its  status  remains  unclear  -^J^g* 
tive/constructivist  views  on  expertise  and  professional  problem-sol ^n?.  In  spite  of 
these  difficulties,  I  believe  that  there  can  be  such  a  thing  as  a  construct, vn  t  tiiewy  of 
instruction.  At  the  paper's  conclusion,  I  recommend  several  changes  to  revitalize 
ID  theory  and  its  place  within  practice  and  within  the  discipline. 

The  CooDerative  Metaphor  „  ... 

Ken  Komaski,  director  of  the  EPIE  Institute,  raised  an  issue  a  few  years  ago  that 
has  conSued  o  affect  my  views.  Citing  Buchanan's  (1963)  study  of  ancient  Greek 
and  medieval  thought,  Komaski  suggested  that  technologies  can  be  divided  into  two 

^ifthe  firT^roup  are  the  arts  practiced  on  matter  and  on  the  many  things  and 
Zcel Sundln  nature.  These  arts,  such  as .sculpture J.  ^^J^^g 
pain  tine  carpentry,  cooking,  etc.,  are  arts  in  which  the  form  in  the  artist  s 
Sd!?o'u  d  be  impressed  on  the  matter... which  could  be  fashioned  and  formed 
TBuchanan,  1963]."  [O  Jur  uses  of  all  technologies  have  been-and  contoue  to  be_ 
influenced  by  this  view  of  "technology  as  a  system  of  exploitation.  (Komaski, 

1987,  p.  9;  italics  removed) 
The  exploitative  technologies  are  contrasted  by: 
those  arts  "practiced  on  human  beings,  who  also  have  artistic  capacities.  In 
humans  there  are  "natural  processes  which  if  left  to  themselves  migh ^  accom- 
plish their  ends,  but  if  aided  by  the  professional  would  accomp  ish  their  ends 
more  easily  and  more  fully.    Medicine  and  teaching  were  the 
discussed  examples  of  such  arts.  They  were  called  cooperative  arts  because  they 
were  understood  to  be  cooperating  with  rational  natures."  The  physician  who 
gains  the  cooperation  and  confidence  of  patients,  and  the  teacher  who  gains  the 
Sg  cooperation  of  students,  are  much  more  apt  to  end  up 
and  competent  learners  than  those  doctors  and  educators  who  fail  to  gam  such 
cooperation.  (Komaski,  1987,  p.  9)  ,  .  , 

Komaski  notes  that  "it  is  the  exploitative  technologies  wit* t  their  undeniable ,  and 
demonstrable  efficiency  and  effectiveness,  that  have  shaped  our  thinking  about, 
and  our  practices  of,  all  technologies-including  those  such  as  medicine  and  teach- 
?ne  that  presumably,  function  more  effectively  when  practiced  as  cooperative  tech- 
noi^M£rski,yi987,  pp.  9-10;  see  also  Mumford's  [1967]  distinction  between 
authoritarian  and  democratic  technologies). 
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It  would  be  an  interesting  exercise  to  classify  various  known  educational  tech- 
nologies on  the  exploitative/cooperative  continuum.  I  am  afraid  that  traditional  ID 
theories  fall  too  neatly  into  the  exploitative  category,  with  attendant  consequences. 
The  despised  factory  model  of  schools  has  a  close  cousin  in  the  machine  model  of 
ID.  The  content  of  current  ID  theories  also  belies  this  orientation.  In  Reigeluth's 
green  book  (Reigeluth,  1983b),  only  the  Gagne  chapter  and  Keller's  ARCS  chapter 
treat  student  attitudes  in  any  lengthy  way.  ID's  manipulative  bent  is  ironic,  given 
the  recent  emphasis  on  working  with  schools  in  restructuring  initiatives  (eg, 
Banatny,  1991)  and  on  cognitive  apprenticeships  (e.g.,  Wilson  &  Cole,  1991)  We 
clearly  need  more  cooperative  metaphors  and  rhetoric  in  our  ID  literature. 

Task/Content  Analysis  and  the  Nature  of  Knowledge 

Does  all  learned  (or  taught)  knowledge  have  to  be  pre-analyzed?  Of  course  not 
lnere  is  much  learned  within  any  instructional  environment  that  goes  far  beyond 
the  instructional  objectives.   Curriculum  theorists  and  media  critics  have  been 

7o2\nBv ^  mP*v     •  /  yC^rS  (Hidden  curriculum  reference;  Hlynka  &  Belland, 
UWU  Yet  ID  theorists  and  practitioners  give  every  indication  that  their  method  of 
slicing  up  the  world  is  the  method,  and  that  the  content  resulting  from  their  analy- 
sis is  Mm  content  to  be  taught  to  students.  More  than  anything  else,  this  aspect  of  ID 
theories  has  troubled  me  (see  Wilson  &  Cole,  1992  for  a  similar  discussion), 
liisner  (1988)  puts  the  counter  argument  succinctly  in  an  abstract  to  a  paper: 
Knowledge  is  rooted  in  experience  and  requires  a  form  for  its  representation, 
bince  all  forms  of  representation  constrain  what  can  be  represented,  they  can 
only  partially  represent  what  we  know.  Forms  of  representation  not  only  con- 
strain representation,  they  limit  what  we  seek.   As  a  result,  socialization  in 
method  is  a  process  that  shapes  what  we  can  know  and  influences  what  we  value 
At  base,  it  is  a  political  undertaking,  (p.  15) 
The  conceptual  schemes  we  apply  to  the  world  constrain  that  world.  Similarly,  the 
schemes  instructional  designers  apply  to  content  constrain  and  shape  that  content 
necessarily  distorting  it  to  fit  our  preconceived  notions.   If  by  some  chance  the 
InlT^- a' commun'ty  were  t0  ag™e  on  knowledge  categories,  then  there  might  be 
some  basis  for  using  those  consensual  categories  in  content  and  task  analyses.  But 
t,«^?^°\Tea  ,  Alexander'  Schallert,  and  Hare  (1991),  in  a  recent  review, 
found  25  distinct  knowledge  types  cited  in  the  educational  literature  on  language 
and  cognition.  And  that  article  was  an  attempt  to  simplify  the  problem! 
*  JnZ^T        c,at^°ri"  are  faulty  (which  they  are),  then  how  can  we  design 

£  ^  anSWCr  iS'  d0n"t  make  the  quality  of>ur  instruction 
rise  or  fa  1  on  the  quality  of  your  analysis.  There  is  more  to  instructional  design 

rnlSo'nrfS"-! Bu.ndfrson„m«de  essentially  the  same  point  years  ago  in  his  dis- 
cussion of  the  "lexical  loop"  (Bunderson,  Olsen,  Gibbons,  &  Kearsley,  1981).  The 
lexical  loop  refers  to  the  parade  of  print-to-print  translations  we  put  content  through 
as  part  of  a  traditional  design  process,  beginning  with  needs  and  content  analyses 
and  ending  ,n  paper-based  tests  (see  Table  2).  As  an  alternative,  Bunderson  pro- 
IZ'tLT'l °f  qua!itative  "work  models"  progressing  in  difficulty  and  fidelity  to 
the  target  setting.  These  work  models  or  learning  environments  are  highly  remi- 
niscent of  White  and  Frederiksen's  (1986)  progression  of  practice  environments 
based  on  careful  cognitive  task  analysis  of  mental  models;  I  believe  that  such 
analyses  remain  highly  relevant  to  the  design  of  instruction,  particularly 
multimedia  products. 
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Knowledge  of  the  Master 


Translation  to  goal  statements 
through  goal/job  analysis. 


Translation  to  objectives  list 
through  task  analysis. 


Translation  to  print-based  tests 
through  test  item  technologies. 


Translation  to  print-based  media 
using  text-design  principles. 


Student  expected  to  transfer  text  ma- 
terial into  skills  ot  the  master. 


Wft1*  Model* 

Master  performance  is  documented 
through  multiple  media. 

Wor..  models  are  designed  of  progres- 
sively increasing  difficulty. 

—Learning  environments  simulate  real- 
life  environments. 

—Students  practice  holistic  as  well  as 
parts  skills. 

—Authentic  tools  are  available. 

—Info  can  be  accessed  through  job  aids, 
help  systems,  and  other  resources. 

—Coaching,  mentoring,  and  peer  consul- 
tation is  available  as  needed. 

Students  complete  work  models  l...n. 

Student  demonstrates  master's  knowl- 
edge/skill in  real-life  performance  envi- 
ronment. 


(Actually,  negligible  transfer 
occurs  to  everyday  life.) 


Knowledge  of  the  Master 


TdbUBl  Two  paths  from  master?  to  mastery  (derived  from  Bunderson,  et  aT,  1981. 
p.  206). 

The  role  I  am  advocating  for  analysis  is  fairly  modest.  Analysis  provides  an 
overa  l  f  amework  for  instruction,  and  provides  extra hilp  "j^^^ 
such  as  identifying  likely  misconceptions  or  previous 

u  students'  efforts  to  understand  the  content.  The  role  of      des.gner  s  then  to 
design  a  series  of  experiences-interactions  or  environments  or  £oducta 
intended  to  help  students  learn  effectively.  Neither  the  instruction  nor ^the  wsess- 
ment  o Earning  can  be  as  confidently  dictated  as  they  were  thought  to  be  n  th  e 
5?  But  the  important  point  to  keep  in  mind  is  that  the  design       »  n*  1^ 
such  a  revised  system;  the  design  still  happens,  only  it's  ™JJ 
holistic  more  reliant  on  the  cooperation  of  teachers  and  materials  and  learners  to 
^^.7  fill  in  the  gaps  left  gaping  by  the  limitations  of  our 'J*^*^ 
Instruction  thus  construed  becomes  much  more  integrally  con^ 
and  the  surrounding  culture.  ID  thus  becomes  more  truly  systemic  in  the  the  sense 
that  it  is  highly  sensitive  to  the  conditions  of  use. 
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«j£  "°  Wh0se  2X3  scheme  you  U8e» the  w°rld  doesn't  always  fit 

ep,"emolo6,c»1  categories.  Force-fitting  people's  expertise  into  ID  tax- 
onomies sometimes  can  do  more  harm  than  good.  My  recommended  alternative  in 
not  to  throw  away  the  taxonomies  entirely,  but  simply  to: 
1.  admit  the  tentativeness  of  any  conceptual  scheme  applied  to  content 

EiS  -at  1°  mattCr  h-°W  th0TOUgh  the  tesk-analysi«  net,  it  doesn't  come  close 
to  capturing  true  expertise; 

3"  Sould  Xfflv^r1  rePresentaution  is  80  tentative,  designed  instruction 
mZ^iff  \  '  ,"formatlon-"ch  experiences,  allowing  opportunities  for 
mastery  of  un-analyzed  content; 

4'  contTntr11^      '        °f  fit  betWee"       conceptual  8ch««e  and  any  given 

5-  inXht^zir^3  of  lack  of  fit  can  be  the  most  criticai  to  understand- 

6'  demands*  °"       l0°k°Ut  f°r  th°Se  CritiCaI  P°ints  °f  idiosyncratic  content 

exZ£^JnJ*h  f  C°n?nt  an?y8is  is  less  a  levelin*  exerci8a  a"d  more  an 
exploration  of  the  terrain,  noting  and  even  exploiting  rough  spots.  Rules  verbal 

ntI°tuTaflnHn,Ha,nd  °ther  8ucVate*>ries  <*™  to  have"  such  litiral ep?st  moloS 
Hl^A  andJbetCome  mTere  tools  in  the  design  process.  The  change  is  largely  one  of 

^ruL^rth1  rrsonal,y  do  not  write      *ped  *£^2TsfiSi 

rTtell  n  «  K  sponsoring  agency.  I  try  not  to  teach  each  content  type  sepa- 
naveVdvo;^^ !  apart/romlskills):  I  do  what  Gagne  and  Merrill  (199?) 

SSU^^  A,,«  h0W6Ver'  8h°Uld  be  aWe  *  fit^hin  ^ 

The  Mystery  of  Expertise 

prots°s: 8tmCtiViSt  ViCW  °f  knowledee  Ieads  t0  an°ther  dilemma  central  to  the  ID 

matte{  ?Pert?  ***  **"  intent  best  but  often  have  least  access  to  it. 
^^J^K^  gCtS  auto™t-d,  conscious  representation  typically 
The  corollary  is  just  as  disturbing' 

Tdee8plybey0nd  ru/e  W  ID>  a  Signer  needs  to  know  content 

of  2S™  !b°,Ut  the  many  kinds  of  ^Phisticated  knowledge  required 

Th  s  tvnTclllv  /  ?  t0  kn°Wing  thC  COntent'  th<*  must  kn°w  how  to  teach  tt 
same XS£ tM Z v  C?^au^"tica1^  but  only  after  years  of  teaching  the 
Umi?1  (^erh,ner'  1986)-  D«igners  who  script  instruction  from  a  newlv 
'  "?rfiCial  C°ntent  k"owledge  cannot  be  expected  to  find  us^tSe  right 
analogy,  just  the  right  way  of  approaching  a  topic.  They  do  have  an  advantage"  ver 

tZSSfifi"*  6XPtrt  in  that  th6ir  u"derstanding  of  the  sSTfresWy 
acqu  red;  that  means  they  have  greater  access  to  strategies  that  worked  forXm 
But  this  advantage  applies  only  to  initial  learning  levels Once  inaction  movTs 
to  non-elementary  tasks,  the  designer  is  on  more  wobbly  ground lacks  the 
Zfcl  beflled  V6ate  ? 6  b6St  tCaChing  meth0ds  (if  s^  ^hlng  indeed  ex  s  s) 

^^mfs  a  mea°nn8  0?°T  deSig"  ConceptS  is  not  the  optimal  desijn  solution  but 
Decomes  a  means  of  only  resort  in  such  situations. 

Together  these  two  problems  pose  a  formidable  paradox  for  ID-  The  Deonle  who 
know  the  subject  best  often  can't  relate  to  the  learner,  while  thTdesigner  with  a  wod 
general  schema  for  teaching  and  a  closer  feel  for  the  learned  nX/wt  f^vl 
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a  feel  for  the  subleties  of  the  content  and  is  thus  left  to  deal^^ia^_i8  a  8ub. 
The  role  of  the  knowledge  engineer-the  designer  s  counterpart ™J^**$£ 

des^er  h"  good  peop'le  skills,  asks  the  right  questions  pushes  the 

and  presto!-out  comes  the  expertise,  on  paper  nc  less   But  »fc;«rae™^rn" 

Te  sample    The  level  of  communication  between  SME  and  designer  needs  to 

move  beyond  superficialities;  somewhere  between  the  two  of  ^ 

at^Ss  of  meaning;  the  design  process  must  be  de«%C^^LS 

design  to  occur.  I  personally  don't  know  how  tins  happens.  It  remains  a  mystery. 

Instructional  Strategies  for  classifying 

ID  needs  a  richer  language,  a  deeper  conceptual  framework  fo  L^8"1^ 
instructional  strategies.  While  this  is  not  the  place  for  such  an  effort  I  have 
me  e^y  itched  in  Table  3  a  number  of  instructional :  strategy hat  seem  to 
facilitate  more  active  construction  of  meaning  (see  also  Wilson  &  Cole,  1991). 


Simulations 

Strategy  and  role-playing  games 
Toolkits  and  phenomenaria 
Multimedia  learning  environments 
Intentional  learning  environments 
Storytelling  structures 
Ca*e  studies 
Socratic  dialogues 
Coaching  and  scaffolding 
Learning  by  design 
Learn  by  teaching 

Group,  cooperative,  collaborative  learning 
Holistic  psychotechnologies 


Table  3.  A  sampling  of  alternative  instructional  strategies. 

A  defender  of  traditional  ID  could  suggest  that  our  present  ^n^*jJ 
contains  prescriptions  for  designing  all  of  the  above.  I  would  on* 
of  the  above  strategies  deserves  its  own  mini-design  model,  and  that  many  tradi 
Sona!  design  concepts  seem  only  to  get  in  the  way^  It  seems 
to  discuss  simulation  design  using  terms  such  as  "expository,  inquisitory, 
-synSLr"  and  "summarized    A  new  framework  and  accompanying  Ian- 
guage  is  needed. 

^^rfi^Z™*  reported  some  interesting  ^^^^ 
cians*  beliefs.  About  half  of  those  surveyed  believe  in  close  alignment  of  tests _  and 
Suction  and  careful,  focused  teaching  of  tested  content.  Sh epard  argues  how- 
ever that  such  beliefs  correspond  to  a  "behaviorist  learning  theory .which  requires 
^uential  mastery  of  constituent  skills  and  explicit  testing  of  each  learning  step 
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(p.  2).  She  argues  for  a  constructivist  alternative  that  emphasizes  more  authentic 
methods  of  assessment  such  as  interviewing,  observations,  and  holistic  task  per- 
formances (see  Perrone,  1991;  Linn,  Baker,  &  Dunbar,  1991). 

Jonassen  (1991a)  takes  what  I  consider  to  be  a  more  radical  stance  towards 
assessment,  extending  Scriven's  (1973)  notion  of  goal-free  program  evaluation  to 
the  goal-free  assessment  of  student  outcomes. 
Constructivistic  outcomes  may  be  better  judged  by  goal-free  evaluation  method- 
ologies. If  specific  goals  are  known  before  the  learning  process  begins,  the  learn- 
ing process  as  well  as  the  evaluation  would  be  biased....Criterion-referenced 
instruction  and  evaluation  are  prototypic  objectivist  constructs  and  therefore  not 
1991b)1  °bjectivistic  envir°nments.  (p.  29;  change  based  on  Jonassen, 

I  take  exception  to  ^onassen's  position  for  reasons  I  discuss  below,  but  I  am 
ambivalent  about  the  general  issue  of  assessment.  Surely  ID  theory  makes 
assumptions  that  make  some  people  feel  uncomfortable,  for  example,  that  instruc- 
tion is  purposive  and  goal-directed,  and  that  attainment  of  those  goals  can  be 
assessed.  I  believe  these  assumptions  can  be  wholly  compatible  with  a  construc- 
tivist philosophy;  here  I  differ  with  Jonassen.  And  certainly,  assessment  need  not 
go  to  the  extreme  of  being  goal-free.  It  seems  that  a  key  question  to  ask  is,  Does  the 
test  performance  require  all  of  the  contextualized  reasoning  and  performing  that 
the  target  performance  would  require?  (Frederiksen  &  Collins,  1989).  The  ques- 
tion is  one  of  fidelity  and  validity,  not  of  goal-directedness. 

I  do  not  have  a  problem  with  the  idea  of  goal-directedness  and  measurement,  but 
I  do  have  a  problem  with  how  it  is  often  done.  A  true  mastery  model  with  micro  lev- 
els of  assessment  would  only  be  appropriate  with  highly  defined  technical  content;  I 
believe  such  methods  would  rarely  be  appropriate  in  public  schools,  though  some- 
what more  often  in  training  settings.  Instructional  designers  in  both  school  and 
corporate  settings  feel  the  need  for  better  assessment  methods  that  can  be  more  fully 
integrated  into  the  performance  and  instructional  systems.  I  have  not  sorted  out  all 
the  issues;  I  raise  the  question  because  of  its  clear  importance  to  the  field  and  to 
society  at  large.  I  continue  to  believe  that  improved  assessment  methods  can  be 
developed  that  are  more  consistent  with  a  constructivist  framework. 

Concluding  Thoughts 

The  central  issue  of  this  paper  is  this:  Is  a  constructivist  theory  of  instructional 
design  possible?  and  if  so,  Whai  might  be  it  be  like?  In  typical  backwards  fashion, 
let  me  return  to  the  question  of  the  nature  of  ID  theory  by  defining  terms 

InstructM-teacherts)  and  student(s)  in  interaction  trying  to  learn  something. 
Together  they  form  an  instructional  subsystem  within  a  larger  system  and  com- 
mumty. 

Design-rthe  proces.  which  things  are  made...designers  make  representa- 
tions of  things  to  be  built"  (Schon,  1990,  p.  110).  Design  is  always  done  within 
constraints. 

Constraints  inherent  in  the  design  of  instruction  include- 

a.  Some  situations  don't  even  have  teachers  per  se.  Thus  the  teacher  may  be 
broadly  thought  of  as  the  guiding  agent,  directing  the  learner  toward  accom- 
plishing the  goals  of  instruction.  Learners  themselves  often  function  in  the 
teacher  role,  as  do  instructional  materials  and  programs 

b.  Designers  can  usually  exert  more  direct  influence  on  the  teacher  side  than  on 
the  learner  side  of  the  instructional  system.  At  the  same  time,  designers 
ignore  considerations  of  the  learner  at  their  peril. 
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c.  Designers  can  usually  exert  more  direct  influence  over  materials  and  tools 
than  over  the  interaction  between  learner  and  teacher. 

d.  At  the  same  time,  the  nature  of  the  precise  interaction  betwetn  learner  and 
teacher  is  at  the  heart  of  understanding  instruction. 

These  constraints  pose  dilemmas  for  constructivist  designers.  As  much  as  we 
may  like  to  focus  on  individual  learners'  cognitions,  often  our  access  is  extremely 
indirect  and  limited.  This  also  helps  to  explain  why  ID  is  often  a  goal-based,  stim- 
ulus-design oriented  endeavor,  working  within  a  noisy  system  that  is  near  chaos 
It  is  no  wonder  that  we  get  no  respect;  my  response  to  critics  who  think  we  should  get 
entirely  out  of  such  a  messy  business  is,  "If  we  didn't  do  it,  somebody  else  would. 
Practitioners  of  ID  need  somewhere  they  can  turn  to,  and  our  theories  are  as  good  as 
the  next  guy's."  (1  am  actually  serious  about  this.) 

I  have  become  convinced  that  a  field  is  largely  shaped  by  the  central  questions  it 
puts  for  itself.  In  the  case  of  ID,  the  questions  are  tough;  there  are  no  good  answers, 
the  best  we  can  do  is  put  forth  some  best  guesses.  We  pay  attention  to  stimulus 
design  (a  dirty  word  among  cognitivists)  because  we  have  no  choice.  We  Prescribe 
general  principles  of  message  and  interface  design  because  those  are  aspects  of  the 
instructional  system  that  lie  somewhat  within  our  power  to  influence.  For  good 
reasons  we  tend  to  get  beat  up  by  cognitivists,  constructivists,  and  humanists;  yes, 
it's  a  dirty  job,  but  somebody's  got  to  do  it! 

The  next  generation  of  ID  theory  needs  to  better  fit  the  needs  of  the  practitioner. 
.  ID  theories  need  to  be  thoroughly  grounded  in  a  broad  understanding  of  learn- 
ing and  instructional  processes.   That  foundation  needs  to  be  continually 
evaluated  and  revised.  -.„„„ 
.  We  need  more  modest  principles  that  designers  can  flexibly  apply.  These 
principles  should  be  generic  and  principle-based  in  order  to  to  be  relevant  to  the 
wide  variety  of  situations  encountered  in  everyday  practice. 
•  In  addition  to  generic  principles,  we  need  specific  heuristics  for  dealing  with 
recurring  problems  and  situations  in  ID  practice.   In  particular,  heuristics 
should  be  developed  that  are  sensitive  to: 
—setting  (schools,  business,  museums,  etc.) 
—media  (computer,  instructor-led,  workbook,  etc.) 
—product  type  (stand-alone  product,  program,  system,  etc.) 
— resources  (time,  money,  constraints). 
.  ID  theories  need  to  reflect  a  view  of  ID  as  a  profession.  Designers  need  sophis- 
ticated schemas  of  design  that  go  beyond  the  "technically  rational  models 
presently  available.  Students  should  be  encouraged  to  develop  personal  mod- 
els of  action  through  extensive  practice  in  authentic  settings  with  coaching  and 
opportunities  for  reflection.  (Note  the  theme  of  reflection  running  through  stu- 
dents, teachers,  and  designers.) 
Let  me  conclude  by  drawing  a  parallel  with  a  couple  of  other  disciplines,  est  we 
think  that  we're  alone  in  this  chaotic  science  of  design.   Artificial  intelligence 
rieht  now  is  facing  some  of  the  same  crises  we  are  confronting.  So  is  the  field  ot 
human/computer  interface  design.  Both  of  these  fields  have  a  strong  tie  to  earning 
and  cognitive  psychology.  Both  rely  heavily  (at  least  in  theory)  on  user  testing  and 
field  validation.  Old-time  AI  theorists  presently  are  being  challenged  by  connec- 
tionists,  who  believe  that  parallel  processing  via  neural  networks  is  a  more 
promising  way  to  go  than  symbolic  manipulation  of  IF-THEN  rules.  The  contro- 
versy goes  beyond  symbolic  versus  networked  processing.   A  growing  number  ot 
AI  researchers  have  lost  faith  in  traditional  views  of  the  representabihty  of  knowl- 
edge- the  "situated"  movement  within  AI  goes  so  far  as  to  deny  that  knowledge  is  a 
structure,  and  that  memory  is  anything  more  than  process  (Clancey,  1991;  Brooks, 
1991)   The  field  of  interface  design  is  going  through  similar  growing  pains,  mov- 
ing from  a  screen-design  view  of  the  field  to  a  global,  holistic  view  of  the  entire 
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human  experience  with  computers  (Laurel,  1990,  1991).  This  revised  view  of  inter- 
face design  has  more  to  do  with  human  activity  than  icons,  buttons,  and  windows, 
more  with  dramatic  metaphors,  agents,  and  virtual  worlds.  And  interface 
designers  are  moving  away  from  controlled,  linear  design  models  toward  rapid 
prototyping  and  participatory  design  (B0dker  &  Grflnbaek,  1991). 

I  don't  doubt  that  AI  will  be  around  in  twenty  years.  Nor  do  I  doubt  that  people 
will  still  be  studying  how  computers  and  humans  work  together.  The  theories  and 
concepts  may  change,  but  the  basic  questions  are  still  there;  they  cannot  be  ignored. 
Likewise,  ID  will  not  go  away;  this  is  because  the  questions  behind  the  field  are 
genuine  questions.  The  challenge  for  ID  theorists  is  to  continue  to  adjust  our  theo- 
ries, or  to  replace  them  with  better  ones.  The  goal  is  to  make  our  theories  worthy  of 
the  questions. 
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EVALUATING  THE  IMPACT  OF  TECHNOLOGY 
AT  PEAKVIEW  ELEMENTARY  SCHOOL 
Brent  G.  Wilson,  Roger  Hamilton,  James  L.  Teslow,  &  Thomas  Cyr 
University  of  Colorado  at  Denver 

£lSOme,hi"8  !PC?ial  iS  haPPenln8  at  Peakview  Elementary  School. 
Peakview  is  a  new  school  that  is  implementing  a  number  of  organization!  and 
teaching  strategies  advocated  by  the  school  restructuring  reform  mo  ™menf 
Among  those  strategies  is  the  infusion  of  more  than  80  nf  tworked  ^com- 
putersand  related  technology  and  software.  This  evaluation  smdyTxaSdthe 
impact  of  the  technology  on  the  school  community.  Using  a  !££?Td£ 
collection  mstruments  (e.g.,  classroom  observation,  surveys  and  inSviews  of 
school  personnel  and  students),  we  found  consistent  evidence  L  iSStow 

EST a^ent,al in  fadUtatinS  the  school's 8°a,s'  ^  technolo^teS 
hvely  affecting  student  learning  and  attitudes.  Teachers  are  using  the  tedSl 

and  m  rdUal  StUdCn,tS'  nCeds  and  interests' and  t0  inc  ™*  the  amouS 
and  quality  of  cooperatwe  learning  activities.  Students  use  the  technology 

^Z  lf0Tf  'T^  and  WriUng  3Ctivities' 35  weU  as  for  instructiorS 3 
m  a  variety  of  sublet  areas.  Technology  has  changed  the  way  teachers  work 

^SCt,°na,1JLand  P™fesiona»y'  ^«ng  in  a  net  increai  of  nours  and  at' 
2US££?  *7T  P™*"?**'  effectiveness,  and  saHsfaction.  A  number  o 
S5?T^  3,16  idCntified  as  ^buting  to  the  success  of  PeakvL? 

use  of  technology.  These  factors  form  the  basis  of  a  set  of  re«,mn^ndations  for 
implementing  technology  successfully  in  other  schools.  T^T^r  is  a  brie 

S^Sf  °f  ^T0^  m°rc  C°mp,et^  "ndin*s  are  reP°rt^  n  Krt  ,epTrt» 
(about  50  pages)  and  the  full  report  (about  200  pages  plus  appendices.)  ^ 

Peakview  Elementary  School  opened  its  doors  to  students  in  the  fall  of  1991  Fmm  (hD 
outset,  school  staff  intended  Peakview  to  reflect  concepts  Z^IS 

T^r^T^1  Strat?ieS  indude  mul«-agfng,  teacher  teara  2o!s 

els,  and  a  comm.tment  to  problem-solving  and  cooperative  learning  activUies  A  kev 

component  of  the  reforms  was  a  greater  role  of  technology  to  suZrt  cfasIroSn  aJv7 

™rpfhT         31  "ST™*  in  C°mPUter  and         ™»™  w«  nude  r^ulSTn 
clas^l  na       COmPutersavai,aWe  in  the  school,  most  of  them  d  sauted  £ 
m  each  ^mS.PreSenUy  h0USC  an  aVeraSe  of4^  color  Macintosh  compuT- 

av^able  m^lLVnf^  h  "IT* in  the  I""1"*  and  m*"*  "Computers  typically 
available  in  elementary  school  classrooms.  Technology  products-indudine  ontirZ 

^^COmPUter-baSed  ins^c«— ^ve  repla^science,  .oSSSjS 

rtuirls^S^S^r^  ty  Pcakvi°W  Staff  are  structural  nature  and  do  not 
S?n  ,ficantadd'h°nal  resources.  The  increased  reliance  on  technology  for  instruc 

'    of  me  SoS??  ^ teChn°,0gy  SUpP°rt  the  innovative  s*™ctures  goals 
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A  parallel  question  relevant  to  district  decisionmakers  is: 

.    Would  the  Peakview  use  of  technology  be  a  model  worth  disseminating  to 
other  eLmentary  schools  in  the  district? 

£om SaSTbeing  undertaken.  That  is,  does  the  use  of  technology  «V^SS  Z 

de"  S  the  added  costs  involved  provide  a  justifiable  return  on  investment 
The  Study's  Design 

This  is  Drimarilv  a  case  study  of  Peakview  Elementary  School  and  its  use  of  techiwlogy. 
A^uX^Sata-cSSrtion  instruments  were  used  to  help  provide finable  mformabon 
coning  the  school;  these  ana  discussed  in  the  Method  s^on  below.  The  study  rebed 
heavily  on  written  surveys  and  interviews  of  teachers  and  students. 

The  present  study  was  designed  and  inducted  to  be  a  sort  of  "snapshot"  *  endow 
«pE  Toprovide  a  context  for  understanding,  companson  were  made  of  two 

kinds: 

Beginning  vs.  end  of  school  yea,  Survey  data  were  <^««%£X^ 
August  1991-one  month  after  Peakview's  opemng-and  May  1992,  to^a«™° 
fhe School  year.  This  allows  some  perspective  on  change,  over  the  course  of  the  school 

Peakview  versus  other  schools.  To  gauge  in  what  respects  Peak^trtomSriso" 
schools  in  the  district,  three  additional  elementary  schools  were  selected  for  c^nson. 
Two^h^te  were  selected  primarily  for  logistical  convenience:  Summit  and I  Polton  had 
2ff  SSeiHS^stadenl.  within  UCDs  Division  of  Instructional  Tedroriogy. 
ThLsTff  members  agreed  to  collaborate  with  us  in  conducting  the  research.  Summi 
KSSSSSf  of  Apple  llgs  computers,  and  very  few c— 
rlassrooms  Polton  also  has  a  lab,  with  a  few  computers  in  classrooms.  Parallel  survey 
daS^coE  at  these  additional  elementary  schools;  no  other  data  were ^ected 
From  these  schools.  Dry  Creek  was  selected  because  it  was  PJJJ^J^^ 
Peakview  in  that  computers  were  integrated  into  classrooms,  but  different  because  the 
computers  were  Apple  H's  rather  than  Macintosh  computers. 

Shifts  in  Teaching  Methods 

Collins  (1991),  a  noted  cognitive  psychologist,  cited  eight  trends  in  changing  teaching 
iSSf  Ttai  changes^  supped  by  research  ^^^gL^m 
nniix;  that  each  of  these  chances  in  teaching  method  can  be  facilitated  by  tecnnoiogy.  we 
^S£S^t^^«%  with  a  Irief  comment  relating  the  teaching  method  to 
technology. 

1  A  shift  from  whole-class  to  small-group  instruction.  Gearhart,  Herman,  Baker, 
mffier,  and  Novak  (in  press)  observed  a  dramatic  decrease  in  teacher-led  acui- 
ties when  computers  arc  used,  from  70%  to  less  than  10%. 
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Traditional  Teaching  Mffthlttif 

Whole-class  instruction 

Lecture  and  recitation 

Working  with  better  students 

Less  engaged  students 

Assessment  based  or.  test  performance 

Competitive  social  structure 

All  students  learning  the  same  things 

Primary  of  verbal  thinking 


Technology-AssistpH  hfcthnHc 
Small-group  instruction 
Coaching 

Working  with  weaker  students 

More  engaged  students 

Assessment  based  on  products, 
progress,  and  effort 

Cooperative  social  structure 

Different  students  learning  different 
things 

Integration  of  visual  and  verbal  think- 
ing 


Table  1 .  Trends  toward  constructive  teaching  methods  facilitated  by  technology  (Collins,  1991). 


2. 


3. 


4. 


5. 


t^t°mJ        md  recitation  *°  coaching.  Again,  Gearhart  and  colleagues  (in 

ESS  mCreaSe in  tCaCherS  SCrvinS as  fa5,ilators  thSdSSS of 

behavior)  when  us.ng  computers,  from  20%  to  50%  of  class  time.  CollinVnSl 

comments:  The  introduction  of  a  third  party,  the  computer,™  o ^tuition 

encourages  the  teacher  to  play  the  role  of  a  coach,  in  much  the  same  waymat  a 

gano  encourages  the  teacher  to  play  the  role  of  a  coach  in  a  ptaiKPfc f» 

rS^Tht'Merban  (1988,a)  f°Und  teaCherS  USin*  firSt^rson  ^stru&fns 

wtao^X^  W  did3CliC  C°nStmCtS  Vy°U  ShOUW  d°  tHiS") 
AMftfrom  working  with  better  students  to  working  with  weaker  students  In 
bad.«onal  classrooms,  teachers  often  carry  on  a  conversation  with  bngte  u 

1ZZ?£  ra,SeJ,hSr  haHd;  teaChm  °ftCn  S,OWCT  »**"■»  t°  avofd  e^r- 

SBfaS  «>  l^ol°^  Pattern  is  reversed:  Schofield  and  ™rban 
0988a)  found  slower  students  receiving  two  to  four  times  more  attention  from 

A  shift  toward  more  engaged  students.  A  number  of  studies  have  demonstrated 

deX  tha  fhP  8  V  C°mpUter  betWGen  daSSes  and  aft*  ^ool  "To  the 
0^0!^^  SUpf°rtS  ,0ng-term  eff°rt  rather  than  short  exer- 
KotTXl ^     '  in  ^  thCy        °Ut  °"  comPuter" 

uclsZ*ZZr7Ti  b?edr  teSt  ^rmance  to  assessment  based  on  prod- 
a^Zf  T^f  ? w  ChCrS  h3Ve  t«ditionally  relied  on  end-of-unit  teste  for 
assessment.  Technology  shifts  assessment  efforts  from  tests  to  effort  and  prraresl 
on  projects,  and  on  the  final  product.  This,  of  course,  poses  new  orobfe^S 
teachers  as  they  search  for  meaningful  and  reliable  ways  $™Z£Zo™£. 
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6  A  shift  from  a  competitive  to  a  cooperative  social  structure.  A  number  of 
researchers  have  noted  greater  cooperation  among  students  when  using  technol- 
ogy. For  example,  Harel  (1990)  studied  4th  graders  as  they  developed  their  own 
lessons  to  teach  fractions  to  3rd  graders.  She  found  students  naturally  sharing 
ideas  and  helping  each  other  solve  problems  in  their  programming. 

7  A  shift  from  all  students  learning  the  same  things  to  different  students  learning 
different  things.  A  number  of  studies  have  shown  how  technology  can  support 
students  as  they  tackle  various  parts  of  a  complex  project,  each  contributing  to  a 
larger  final  product.  What  this  means  is  that  students  are  working  on  separate 
aspects  of  a  problem.  Students  working  on  different  learning  goals  can  be  a  logisti- 
cal nightmare  without  technology  to  maintain  focus  and  manage  information. 

8.  A  shift  from  the  primacy  of  verbal  thinking  to  the  integration  of  visual  and  verbal 
thinking.  Visual  media— television,  film,  and  computers— have  begun  to  gain  par- 
ity with  abstract  text  as  a  primary  means  of  learning  in  our  day.  Lectures,  multi- 
ple-choice tests,  and  recitation  of  knowledge  become  less  relevant  methods  when 
faced  with  technology-based  alternatives. 

In  other  words,  society  in  general  and  education  in  particular  are  coming  to  value  a  cer- 
tain approach  to  education.  There  is  some  evidence  that  technology  can  help  education 
practice  move  in  those  valued  directions.  This  line  of  thinking  influence  the  design  of  the 
present  study;  the  reasoning  was:  Technology  can  be  justified  to  the  degree  that  its  use  is 
found  to  facilitate  instructional  methods  and  learning  goals  that  are  valued  by  the  school 
and/or  the  district. 


METHOD 

In  consultation  with  Peakview  and  Cherry  Creek  leadership,  we  developed  a  list  of 
research  questions  to  be  addressed  by  the  study.  These  questions  then  drove  the  devel- 
opment of  data-collection  instruments,  and  provided  a  structure  for  reporting  findings. 
The  questions  were  based  on: 

•  the  stated  goals  and  objectives  of  the  school; 

•  the  expressed  need  of  school  and  district  staff; 

•  the  expected  impact  based  on  the  review  of  literature. 

Instruments 

The  lengthy  list  of  evaluation  questions  suggests  that  a  variety  of  data-collection  methods 
be  used;  hence,  a  number  of  instruments  for  data  collection  were  developed  and  used. 
The  major  data  collection  instruments  are  listed  below. 

August  1991     Baseline  survey. 

May  1992         Main  survey  (administered  at  Peakview  and  3  comparison  schools) 

•  Primary  student  surveys. 

•  Intermediate  student  surveys. 

•  Teacher  surveys. 

•  Staff  surveys. 

May  1992         Peakview  teacher/ staff  interviews. 

May  1992         Peakview  student  interviews. 

May  1992         Peakview  teacher  logs  and  written  reports. 
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Depending  on  the  data  source,  a  number  of  different  strategies  for  interpreting  the  data 
were  used.  For  surveys,  bar  charts  were  developed  to  display  the  means  and  distribu- 
tions of  responses  to  questions;  responses  were  separated  according  to  school,  with 
Peakview  separated  fro.n  the  remaining  schools  to  allow  clear  comparisons.  These  bar 
charts  allow  a  visual  comparison  of  the  response  patterns  between  Peakview  and  the 
comparison  schools.  Line  charts  were  also  developed  comparing  the  mean  responses  of 
the  four  schools.  Where  possible,  chi  square  (X2)  or  analysis  of  variance  (F  statistic)  were 
calculated  to  compare  responses  across  schools. 

Responses  to  open-ended  questions  from  the  survey  and  interview  questions  were 
treated  similarly.  Data  were  coded  into  qualitative  categories;  these  categories  were  then 
used  for  reference  and  retrieval  In  addition,  where  appropriate  counts  were  conducted 
on  the  frequency  of  different  response  categories;  the  frequency  breakdowns  of  these 
counts  are  presented  in  the  findings. 


SELECTED  FINDINGS 

This  section  reports  the  findings  of  the  study  in  highly  abbreviated  form.  Graphs  and 
tables  comparing  Peakview  with  comparisons  schools  are  presented  in  the  full  report. 

Use  of  Technology 

Prior  to  coming  to  Peakview,  teachers  showed  a  typical  range  of  prior  experience  in  using 
technology.  In  August  1991,  Peakview  teachers  most  commonly  reported  "sometimes"  to 
the  statements: 

I  have  used  computers  before  with  children  in  my  classroom. 

I  have  used  computers  before  with  children  in  school  (computer  lab,  etc.). 

Since  coming  to  Peakview,  teachers  find  themselves  using  technology  daily  in  their  class- 
rooms. Peakview  students  spend  roughly  twice  the  time  on  technology  as  a  students  in 
comparison  schools.  Teachers  keeping  a  weekly  log  reported  spending  39%  of  students' 
worktime  spend  using  technology.  Peakview  students  confirm  this  report  of  greater 
technology  use.  Peakview  elementary  students  report  using  computers  several  times  a 
week,  whereas  comparison  students  average  about  once  or  twice  a  week.  Primary  stu- 
dents report  similar  usage  patterns. 

Comparison  to  teachers  at  comparison  schools,  Peakview  teachers  report  significantly 
more  use  of  technology  in  all  major  areas: 

•  word  processing  (p  <  .001 ), 

•  authoring,  (p  <  .06), 

•  art  and  graphic  activities  (p  <  .001), 

•  instructional  software  (p  <  .01 ),  and 

•  laserdisc  viewing  (p  <  .001). 

Teachers  report  greater  use  of  alterantive  media,  greater  use  multiple  modalities. 
Student  self-report  data  corrobate  this  finding.  Contrary  to  sales  pitches,  teachers  report 
working  longer  hours  as  a  result  of  the  technology.  On  the  other  hand,  many  teachers 
report  having  more  time  for  small-group  and  individualized  activities. 
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The  technology  available  at  Pcakview  affects  the  kinds  of  instructional  strategies  teachers 
cKoom  Peakview  teachers  overwhelming  agree  that  tech^ogy  makes 
Z^MnTmore  effective.  Consistent  with  Collins'  (1991)  analysis  of  trends  m  educa- 
£  we  found^hat  technology  enables  teachers  to  succeed  in  areas  that  are  important  to 
them,  summarized  in  Table  2  below. 


kgarnjng  Goal 


Importance 
fo  Teachers 


Accommodate  different  Very  high  priority 
learning  styles 

Self-directed  learning     Top  priority 


Accommodate  students  Top  priority 
working  on  multiple 
learning  goals 

Students  teaching  Very  high  priority 

themselves  and  others 

Student  research  skills    High  priority 
and  independent  access 
of  information 


Effect  of  Class-  Peakview  versus 
rwTn  Technology     ^hpf  fchOPlS 

Moderate  to  heavy  Peakview  higher 
(p<.001) 

Heavy  Peakview  higher 

(p<.001) 

Moderate  to  heavy  Peakview  higher 
(p<.001) 


Moderate  to  heavy  Peakview  higher 
(p<.01) 

Moderate  to  heavy  Peakview  higher 
(p<.01) 


Table  2.  Teacher  priorities  of  different  learning  goals,  and  how  technology  impacts  on  those 
learning  goals. 

Peakview  teachers  report  that  technology  affects  other  desirable  educational  goals  and 
strategies,  including: 

•  cooperative  learning  activities  (whereas  non-Peakview  teachers  reported  veiy 
little  effect  of  technology  on  cooperative  learning); 

•  productive  time  on  task; 

•  student  atten  ti  veness. 

Teacher  attitudes 

At  the  beginning  of  the  19992  school  year,  a  survey  asked  Peakview  teachers  to  contrast 
their  prior  conceptions  to  present  conceptions  toward  technology: 

[I  used  to]...  watch. 
fNowIl...  try. 

llusedto}...     lookfor  the' 'expert"  to  help  kids  who  were  stuck.. 

{Now  II..       try  things  out  for  myself  -and  by  doing  it  daily  several  times,  I  m  learning 
some  procedures  by  heart! 

II  used  to} ...    see  the  potential  of  computers  for  other  people 
INow  I/...       see  the  potential  for  myself! 
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U  used  to]...    avoid  computer  at  all  cost. 

[Now  I] ...      only  avoid  them  during  Bronco  games. 

These  responses  clearly  indicate  a  shift  toward  more  receptive,  positive  attitudes  toward 
technology.  Later  in  May,  20  of  21  responding  teachers  at  Peakview  reported  that  their 
attitudes  toward  technology  in  the  classroom  have  changed  substantially  over  the  past 
year.  Teacher  interviews  yielded  the  same  finding: 

/may  have  been  a  little  skeptical,  at  first,  but  I  am  a  true  believer  in  the  vital  role  computers 
nave  in  our  educational  system!  Nora,  Kindergarten  teacher 

My  god  is  to  learn  morel  I'm  getting  over  a  lot  of  my  "fears"  about  computers,  but  there's 
an  awful  lot  I  still  need  to  learn!  I  feel  a  comfort  level  settling  in,  but  I  need  more  informa- 
tion!!! More  ttme  to  learn!!!  Nora,  Kindergarten  teacher 

What  I  will  always  remember  about  this  year  is  the  realization  that  teachers  need  not  be  com- 
puter wtzards...just  learners.   Matt,  intermediate  teacher 

Peakview  teachers  report  higher  "comfort  levels"  than  their  counterparts  in  using  word 
processing,  videodisc  viewing,  and  arts  and  graphics  programs.  With  some  exceptions 
teachers  at  all  four  schools  reported  relatively  low  comfort  levels  using  databases 
spreadsheets,  and  programming.  Peakview  students  received  more  encouragement  from 
heir  teachers  in  using  technology,  and  said  their  teachers  seemed  to  enjoy  using  the 
echnology.  Virtually  all  of  the  data  we  collected  in  the  study  served  to  corroborate  the 
finding  of  positive  teacher  attitudes  at  Peakview. 

Student  Achievement 

Eighteen  of  22  Peakview  teachers  agreed  that  student  achievement  is  increased 
when  they  use  technology.  The  remaining  four  were  undecided;  none 
disagreed  with  the  statement.  From  teacher  interviews: 

/  think  they  are  excited  about  learning.  It's  a  new  avenue...they  are  doing  writing 
land]  reading  things  I  didn 't  think  first-  and  second-  graders  could  do.  It's  interest- 
ing.I  have  kids  who  are  working  on  projects  land]  units...the  learning  is  more  in 
depth...more  opportunities,  not  iust  a  book  and  paper.  Charlotte,  primary  teacher 

Supermunchers—the  kids  taught  themselves  new  words  so  they'd  be  able  to  do  it 
They  really  have  made  themselves  learn  the  new  words.  Ginny,  primary  teacher 

Achievement  gains  in  reading  and  writing  are  very  pronounced,  especially  with  kids  on 
the  low  end.  Adam,  intermediate  teacher 

I've  never  had  a  class  that  has  known  all  the  letters.  This  year  every  child  in  my  class 
knows  every  letter  of  the  alphabet.  A  lot  of  it  is  due  to  the  computer.  I  can't  say 
exactly,  but  I  feel  certain  that  it  is.  Mary,  Kindergarten  teacher 
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jfrfrnoiogv  iff  a  gwd  wav  to: 


Learn  basic  skills 


Help  students  learn  to  work 
in  small  groups. 

Access  and  use  information 

Learn  oral  and  written 
communication  skills 

Help  students  learn  to 
research  and  report  on  a  topic 

Learn  problem-solving  skills 


Egfllcview 
Tochers 

Agree  to 
Strongly  Agree 

Agree  to 
Strongly  Agree 

Strongly  Agree 


Agree 

Strongly  Agree 
Agree 


Non-Peakview 
Tpachera 

Agree 
Agree 


Agree  to 
Strongly  Agree 

Undecided 

Strongly  Agree  to 
Agree 

Agree 


firhftftl  Differences 


ns 


ns 


ns 


Peakview  higher 
(p<.05) 


ns 


ns 


Table  3.  Teachers  as  a  way  to  teach  different  learning  outcomes. 
Student  Attitudes 

Attitudes  toward  school.  Peakview  students  generally  like  school,  with  a  majority 

tatl^l  is  not  too  easy  and  not  too  hard,  but  rather 
2  2 makes  school  "a  lot"  more  fun.  Peakview  students  report  hkmg ;  school  'a lot 
I^bSiSrfthc  technology.  Non-Peakview  intermediate  students  differ  markedly 
(p  <  .001),  with  responses  averaging  "a  little  more." 

Attitudes  toward  technology.  Peakview  students  were  asked  in  the  August  Baseline 
KiSStos  aimed  at  gauging  their  feelings  toward  the  teehnolopy  availab k 
auSchoT  Intermediate  students  generally  agreed  about  the  a 
use  computer,  about  their  parents'  endorsing  their  learning,  and  that  technology  was  a 
good  way  to  learning  something  new. 

Intermediate  students  at  all  four  schools  uniformly  reported  wanting  to  learr .more, about 
tecZlogy,  with  Peakview  students  showing  markedly  greater  en  hus.asm  <p<.001). 
Primary  Indents  in  focus  interviews  at  all  four  schools  unanimously  agreed  vnth  the 
aTe  statement.  Students  at  the  four  schools  also  concurred  that  learning  about  technol- 
ogy  wis  an  important  goal.  Again,  Peakview  intermediate  students  showed  a  stronger 
conviction  than  non-Pcakvicw  students  (p<.05). 

Students  across  the  four  schools  reported  a  preference  for  technology-based  learning  over 
SttotS  learning.  ^kvleStalernili.te  students  expressed  strong* ^greeinent 
tnan  non-Peakview  students  (p<.05).  This  confirms  intermediate  student  reports  in  the 
August  Baseline  Survey. 

Teachers  confirmed  positive  student  attitudes  regarding  technology. 

They  love  the  computers!  For  Free  Choice  Center,  I  always  have  to  say  "Who  wants  to  go  to 
the  computers  first?"  It 's  the  most  favored  thing  that  they  like  to  do. 
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They  love  its  technology.  They  have  a  kind  of  7  can '  attitude. 

Students  spoke  for  themselves  about  their  attitudes  toward  the  technology: 

Technology  is  really  a  outstanding  thing.  1  hope  1  am  good  in  technology.  We  didn 't  use 
computers  much  in  our  old  school.  Some  of  the  people  in  our  class  are  really  good  typers  on 
thecomputer.  I  really  like  my  school.  I'm  glad  we  have  a  lot  of  computers.  Brittany 

I  used  to  write  and  write.  But  I  never  had  any  pleasure  with  it.  I  would  cherish  the  times  I 
got  to  go  to  the  computer  lab.  I  never  dreamed  of  using  as  cool  technology  as  1  do  now.  Anne 

liS,°l°*y  and  this  school  for  preparing  me  for  tomorrow's  society.  1  am  very 
scared  about  tomorrow,  but  I  am  prepared  and  confident  in  my  peers.  !  wish  to  have  a  future 

Kevin"  markeL  1  "lS0  think  thai  ^  sch^ould  have  thTpMege 

Attitudes  toward  learning.  Peakview  students  were  asked  in  the  August  Baseline  Survey 
questions  related  to  technology  and  learning.  Students  at  all  grade  levelTreport  a£Z 

hSUHl ?  %  y  Wiil  he'P  ,hCm  ,eam-  Peakview  teachere  ^served  that  srudents  are 
highly  motivated  to  stay  on  task  and  leam  with  technology: 

I%nkidS,'!reciUffeCLd;  Cun  lstay  ™  recess,  can  1  stay  late?  We  have  such  a  short  recess 
period,  1  wish  we  had  more  opportunities  to  do  more.  Michael,  intermediate  teacher 

'rt82  t£Jl!L*??  Ch0°Se  t0  $tiy  in  ani  WOrk  ■**  c™V»i«s  rather  than  go  outside  for 
recess .  Nora,  Kindergarten  teacher  ' 

Kids  come  early,  stay  late,  stay  in  at  recess.  Brad,  Kindergarten  teacher 
Students  commented: 

Hike  computers  a  lot  and  I  do  as  much  as  I  can  on  computers.  We've  got  a  computer  at  home. 
Its  a  quicker  way  to  do  things.  It's  fun  and  it's  good  to  learn  with.  Charles  Johnson 

//  we  didn 't  have  technology  everyone  would  be  bored... there' d  be  nothing  to  do.  Mathew 

dents  („< m\ \  vl nterm0d* ,c  s,udon  s  *&KC*  ™™  strongly  than  non-Peakview  stu- 
dents <P<.001).  Primary  students  at  all  four  schools  also  reported  that  using  technotoev 
makes  them  feel  good  about  themselves.  Most  (83%)  report  computers  Sg-^S 

cSv^  '      PO^tive  effects  technology 

Technology  has  really  been  a  very  good  experience  for  me  this  year.  I've  been  gettinv  better 
grades  m  which  I've  been  excepted  into  the  G.T.  prtgram  and  Ithinkifs  dZtilnoll 
because  you  can  learn  stuff  with  technology  like  laserdiscs,  G.T.V.,  and  CD-ROM  I  wiU% 

£3  S  TAUndnRid?l  ^  ym  and  n  have  at  *»'  on  class  (not  courting  aZut 
class)  that  has  at  least  6  computers  in  it  like  Peakview.  1  >ve  done  some  projects  JthZt  tech- 
nology and  some  with  it,  and  it  was  much  easier  with  the  technology.  Charlotte 
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Mv  feeling  about  technology  are...that  since  so  many  computers  are  at  Peakview  I  seem 
sZrter    The  computers  are  like  electronic  textbooks  Except  they  are  tons  more  fun.. 


Elizabeth 


Tntermediate  Peakview  students,  asked  in  August  if  they  were  worried  about  making 
rnsX?onthe  computer,  responded  diversely  The  fact  that  so 
concerns  abc-U  errors  suggests  that,  even  for  children  who  view  romputers  as  easy  and 
view  themselves  as  good  at  computers,  making  mistakes  can  still  be  a  concern. 

A  survey  response  by  a  fourth  grader  further  illustrates  how  many  children  feel  about 
the  technology  at  Peakview: 

I  used  to  not  be  alod  to  use  technology  that  mach  a'  M.  f  felt  relly  dome  when  I  was  atmyM 
school.  But  now  I  think  technology  is  grand  but  in  a  way  it  is  hard.  So  well  1  stell  love  tech- 
nology andhowe  it  work 's.  Heather 

In  summary,  students'  self-concepts  are  affected  by  a  number  of  factors  Trying  to 
Isolated  the'effects  of  technology  is  difficult  The  great  majority  <^^*™™£ 
nology  as  easy,  particularly  Peakview  students.  However,  a  number  rftakww 
students,  at  the  beginning  of  the  year,  reported  worrying  about  doing  something  wrong 
on  the  compute.lt  seems  that  there  may  be  some  students  with  concerns  about :*m 
technology  and  their  confidence  in  using  it.  On  the  positive  side,  students  at  all  four 
sfhool  generally  agreed  that  technology  make  them  feel  good  about  themselves^  Eighty- 
s  x  pe^ent  of  Peakview  intermediate  students  agreed  the  statement.  This  ind.cates  a 
strong  number  of  students  whose  self-concepts  are  likely  helped  by  through  working 
with  technology. 

Student  empowerment.  An  important  educational  goal  is  to  help  children  come  to  feel  in 
control  of  fheir  own  learning^Taking  charge  of  one's  learnin g-^pendent  o f  he 
teacher's  behavior  and  the  school  environmcnt-is  often  not  entirely  achieved  ™hl  h  gh 
school.  Because  technology-based  activities  can  often  take  the  form  of  indepen den  tor 
cooperative  research  activities,  we  were  interested  in  gathering  information  on  tms  ques- 


tion. 


Intermediate  students  across  the  four  schools  generally  agreed  with  the  statement, 
I  like  technology  because  the  teacher  doesn't  always  have  to  help  me .  Primary  students  showed 
a  similar  profile  of  agreement  to  the  statement.  Students  generally  agreed  with  the 
statement  I  like  to  make  my  own  choices  about  how  1  use  the  technology,  although  a  number  of 
students  were  "unable  to  judge."  Primary  students  at  the  four  schools  concurred. 
Responses  were  similar  to  tl.  question,  J  like  to  think  of  my  own  ways  to  use  technology. 

Attitudes  of  children  with  special  needs.  The  motivation  and  attitudes  of  certain  cWldren 
are  especially  important  when  considering  educational  innovations.  For  example,  if  most 
children  had  positive  attitudes  toward  a  new  strategy,  but  low-achieving  children  hated 
it,  that  finding  would  be  cause  for  concern,  even  if  the  strategy  were  generally  beneficial. 
Teachers  were  asked  specifically  about  technology's  potential  in  enhancing  the  self- 
esteem  of  at-risk  students.  Staff  members  at  all  four  schools  agreed  that  technology  can 
enhance  the  self-esteem  of  these  children;  Peakview  staff  members  stiongly  agreed  with 
the  statement. 

Students  limited  physically  seem  also  to  be  helped  by  the  technology  One  advantage  is 
in  the  ease  in  interacting  with  the  keyboard  for  students  who  have  difficulty  controlling 
their  fine  motor  movements.  A  Peakview  special  education  teacher  commented: 
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Technology  has  changed  my  life  and  the  lives  of  my  students,  almost  entirely  with  positive 

much  more  mofnwte*  fcy  such  avenues  as  computers  and  laserdiscs  to  read  and  wrUe.  In 
writing,  for  example,  students  can  pull  up  a  variety  of  pictures  for  instriratioVon  he 
computer,  thenerjoy  the  increase  of  their  keyboarding  skill  L  their  WSZdiZ 

Z  *»  r?, thar  stories- For  students  with   motor  mcuiZ,  S  ZTkSZ 

produce  legible  writing  the  computer  opens  a  whole  new  avenue  of  flexible  ex/ession.  Gerri 
K-5  special  education  teacher  ^  oem' 

ZhBlZt US  """'ft  'he  Tl  U  0"  SUCCCSS  and  ^0Wth  H  &«  child™  they 
wm^ZZX  fT  othr™iemk  "<"■  **«*ng  a  special  needs  child  in  my  c'J- 
roomu t  proof  of  that.  It  ts  through  the  computer  that  he  is  able  to  choose  spelling  words  read 
and  follow  a  book  on  the  CD-ROM  and  most  importantly  be  able  to  comTunShrouTa 

%%;iZn8  T"?  ^  ?""?■  1  kmW  that  05  he  cont™«  ■>  «*  techZhgy  Zwm 

Mm  VZ^0flCK  '  he  WiU  bKOme  a  better  communicator  with  thoYe  around 

mm.  Charlotte,  primary  teacher 

One  teacher  commented  on  lower-achieving  students  and  the  help  technology  can  be: 

lr^!L^l^lbKmeff  Tm  °fwritten  inf°^ion.  I  have  seen  "non 
writers  ,  especially  those  hampered  by  poor  fine  motor  skills,  show  tremendous  pride  in  their 

tZch^t?  Wh°'  eigHt  m°nthS  **°>  WOuld  *«*  ™  «  thTmZoTof 

fig^  fto^Zogy'.  y  PUTSUe        °f  intCreSt  rttngin*  ft™  Amerkan  r»K««* 

Summary  of  the  Findings 

The  table  below  outlines  the  various  effects  we  have  found  at  Peakview  Elementarv  Tho 
table  does  not  include  the  strength  of  evidence  for  the  variou7ind^ 
provrde  a  handy  overview  of  the  various  factors  affected  by  technoSat  thl  cS 
Although  the  study  identified  a  number  of  areas  that  nej refinement  t  cLTd  not 
Wentrfy  a  general  impact  area  where  the  technology  was  perceived  to  have  a  negate 
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ImpactofT^nologyOn:  Singly  ?^ 

Use  of  Technology  * 
Use  of  multiple  modalities  * 
Use  of  media  x 

Impact  on  Teaching 

Accommodating  ability  levels  x 
Small  group  instruction 

Competitive  vs  cooperative  social  structures  J 
Verbal  and  visual  learning  media 
Time  on  technology  J 
Time  on  task  Q 
Self-directed  learning  J 
Accommodating  multiple  learning  goals  a 
Accommodating  learning  styles 
Students  as  teachers  * 
Information  access  and  research  activities  X 
Changes  in  teacher  work 


X 


X 


Teacher  attitudes 
Teacher  comfort  levels 
Computer  coordinator 
Taking  computers  home  * 
Computer  as  a  stimulus  to  change  * 
Teacher  perceptions  of  students 

Self-concepts  as  competent  professionals  A 

X 

Student  achievement 
Attainment  of  basic  skills 

Access  and  use  of  information  X 
Problem-solving  skills 
Oral  and  written  communication  skills 
Researching  and  reporting  X 
Student's  perspective 
Small-group  work 

Other  forms  of  achievement  X 

Student  attitudes  * 

Towards  school  J 

Towards  technology  J 

Towards  learning  J 

Towards  teachers  * 

Towards  self  * 

Student  empowerment  * 

Special  needs  students  x 


Table  4.  Summary  of  the  impact  of  technology  at  Peakview  Elementary  School 
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Conclusions  of  the  Study 
Students  and  teachers  are  using  the  technology.  The  available  evidence 

software,  and  laserdisc  viewing.  Students  use  technology  in  finding  infor- 
mation, researching  and  writing  about  topics,  and  in  problem-solving  activi- 

Technology  is  changing  classroom  practice.  Peakview  teachers  overwhelm- 
ingly  prefer  4-4  computers  in  the  classroom  over  computer  labs.  Technology 
has  simulated  innovation  in  the  way  subjects  are  taught;  several  teachers 
Elff8/;  teaChing  l°  better  inteSrate  ^hnology  into  different 
SE'  3? CrS  rep°rt  3  deSire  10  continue  learning  more  about  the 

technology,  ,n  order  to  continue  changing  their  classroom  practices 

™~ZllT  T10"8  Tre  h°UrS  bea,Uie  of  the  technology,  and  having 
more  control  over  their  work.  6 

Tte  technology  has  changed  teachers'  beliefs  and  attitudes.  Peakview  teach- 
ers underwent  an  attitude  shift  in  their  first  year  using  technology  at 
Peakview.  They  came  to  see  technology  as  a  powerful  tool  to  ££SX£. 
»ng  ,n  elementary  children.  They  believe  thattechnology  can  te  a  veWctefor 

mf ny  °f  ^  ,eaming  and  -^tional^oals  that  are  impor- 
tant to  them      h  as  probiem.soIvi    skU1/  CQ       tj  6  ,earni  jndepen. 

dent  research  skills  and  individualization  according  to  learners'  needs  £ 

leTnobgTes.  C°      CnCe  ^        °Wn  abiIitiCS  10      COmPuters  and  °thS 

fitdu!$  leam  ^ectively  usinS  the  technology.  Students  are  showing  tenta- 
tive learning  gains  in  a  variety  of  areas.  Their  skill  at  using  technology  is 
obviously  improved.  Some  teachers  report  reading  and  vocSry 
mnprovements  in  early  grades.  Students  do  more  editing  and  revSng  of 

u^ZTs  usfmS  w°rd?r0CeSSin8  t00,s-  SPe»  deckers  are  only  sparing^ 
used  by  students.  A  number  of  intermediate  students  are  using  theSchnol- 
ogy  for  a  vanety  of  independent  or  small-group  projects,  including: 
Combining  paint  graphics  and  word  processing; 

•  Incorporating  scanned  and  clip-art  graphics- 

•  Authoring  HyperCard  projects; 

•  Using  CD-ROM  and  optical  laserdisc  information  references- 
^a^COrP°!?ted5D'ROM  and  ,aserdisc  ^uences  into  HyperCard  projects 
^n^^rPtoje?tsr^rt  8reater  StUdCnt  intCreSt  and  ^a«ve  in  CI 

f^ZedZZnTted  t0Ieam  Witk  the  '"Oology.  Students  experience 
rrii  ?™  and  emP°werment  as  a  result  of  the  way  technology 
ers  sLIm  a»  f  I?1™  th3t  StUdentS  WOrk  more  Productively  with  comput- 

•  school  S  affeCted  bv  ^hnology,  toward: 

•  technology, 

•  learning,  and 

•  themselves. 

Technology  is  a  vehicle  for  many  of  the  school's  reform  initiatives  Multi- 

afe^" — — - »— 

'  C3SS25!£  "r"",s  15  fM5"*e  whcn odi,ins  and  revlstoni  - 
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•  Independent  research  can  be  more  easily  accomplished  when  eletfronic 
forms  of  references  are  consulted,  and  when  student  data  is  stored  and 
manipulated  on  computers. 

.  Technology-related  projects  lend  themselves  well  to  cooperative  learning 
groups.  Students  can  collect  projects  into  electronic  portfolios,  allowing 
for  alternative,  authentic  assessments  of  their  learning. 
Each  of  these  initiatives  is  part  of  Peakview's  innovative  philosophy  of  ele- 
mentary education.  There  Is  no  question  that  without  the  technology,  many 
of  these  practices  would  go  forward.  However,  access  to  the  technology 
improves  the  likelihood  that  these  reforms  will  succeed. 
Key  elements  of  successful  implementation  include: 

•  Computers  abundantly  available  in  the  clastroom.  Each  classroom 
houses  4-6  color  Macintosh  computers;  computers  are  often  shared 
between  adjoining  classrooms  to  allow  more  flexible  use  of  resources. 
According  to  teachers,  the  number  of  computers  in  the  classroom,  and 
teachers'  and  students'  easy  access  to  them,  is  a  powerful  factor  con- 
tributing to  successful  implementation. 

•  Shared  commitment  and  vision  of  school  reform  with  technology  as  an 
essential  component.  The  amount  of  work  required  to  successfully 
begin  a  school  with  a  number  of  innovations  should  not  be  underesti- 
mated. The  Pcakview  community— particularly  the  teachers  and  admin- 
istration—articulated a  vision  for  the  school,  and  they  committed  to 
making  that  vision  happen.  The  entire  staff  bought  in  to  the  program, 
and  worked  had  to  overcome  the  many  obstacles  and  challenges 
encountered  along  the  way.  An  atmosphere  was  cultivated  that 
encouraged  offering  mutual  support  and  sharing  resources. 

.  A  supportive  district  and  principal.  Peakview  received  the  support  of 
district  administration  in  developing  an  innovative  set  of  values  and 
methods  for  elementary  education.  The  principal  supported  the  use  of 
technology  at  the  school,  and  enthusiastically  learned  to  use  the 
Macintosh  along  with  the  rest  of  the  staff.  The  leadership  and  commit- 
ment of  district-  and  building-level  administrators  created  conditions 
conducive  to  success  at  the  school. 

•  A  strong  computer  coordinator.  Peakview  has  one  teacher  assigned  full- 
time  to  technology  leadership  and  support.  This  position  seems  to  be  a 
critical  component  of  the  school's  implementation  of  technology.  The 
computer  coordinator  seems  to  give  other  teacher  the  courage  to  "charge 
ahead"  in  the  use  of  the  technology.  Hardware  and  software  systems  are 
maintained  and  managed;  inservices  are  provided  to  staff  and  students; 
troubleshooting  help  is  provided  for  problems  as  they  arise. 

•  Early  and  thorough  teacher  training.  Before  the  school  opened,  teachers 
received  training  on  Macintosh  operating  system,  Microsoft  Works,  and 
instructional  software  to  be  used  in  classes.  Inservice  lessons  have  been 
regularly  made  available  to  teachers  and  students.  This  access  to  exper- 
tise seems  to  have  been  very  helpful  to  teachers. 

•  Taking  computers  home.  Following  initial  training  in  the  spring  of 
each  teacher  was  given  a  computer  to  take  home  for  6  weeks.  According 
to  many  teachers,  this  allowed  them  time  to  become  comfortable  with  the 
technology  before  school  started.  Many  teachers  reported  receiving 
tutorial  help  from  their  children. 

•  User-friendly  systems.  The  color  Macintosh  LCs  at  the  school  have  con- 
tributed to  the  attitude  change  among  many  teachers.  High-quality 
software  is  another  factor  in  the  school's  successful  implementation. 
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2. 
3. 
4. 

5. 
6. 

7. 

8. 


Recommendations  to  the  School 

Continue  inservice  training,  particularly  informal  lessons  with  teachers  and 
students  attending  together. 

Train  teachers  in  uses  of  database,  spreadsheet  programs,  and  other  tools. 
Continue  computer  coordinator  position. 

Periodically  perform  a  self-study  to  assess  progress,  set  priorities,  spot 
trends,  and  establish  strategic  goals  and  plans. 

Build  regular  maintenance  and  upgrade  costs  into  regular  school  budget 
Continue  developing  electronic  portfolios  and  other  authentic  assessment 
methods. 

Develop  improved  assessment  measures  to  track  performance  gains  over  a 
period  of  years. 

SSJ"8  l°  develope,ec,ronic-mai,  (e"ma")  and  telecommunications  capa- 
9.       Continue  to  cultivate  parental  involvement. 

10.  Find  more  problem-solving  software,  particularly  In  science. 

11.  Care fully  implement  cooperative  learning  activities,  ensuring  equitable 
workload  among  students  and  efficient  use  of  time. 

Recommendations  to  the  District 

1.       Use  Peakview  as  a  model  for  other  elementary  schools  in  the  district. 
Perform  a  cost/benefits  analysis  to  determine: 

•  if  P  .kvicw  technology-related  outcomes  are  highly  valued; 

•  lMhc  value  of  those  outcomes  justify  the  additional  cost  of  the  technol- 

3*       ^ff  ?ulCCtiVC  mcasurei»  »f  Peakvlcw's  performance  into  the  data 

provided  by  the  present  study. 
4.       Measure  student  competencies  throughout  the  district 

a^dnmeUthodsSUPPOrt  ******  **  *  ^ ,0  try  °Ut  new  technoIo8^ 
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Thought*,  Feeling*  and  Action* 
Integrating  Domain*  in  Library  Instruction 

Presentation  Handout 

<„♦  J1?'*  P^wnfctiondescribe*  a  framework  for  library  instruction  which 
integrates  learning  in  the  cognitive  and  affective  domains.  Central  to  this 

SJ^ifSSy*  ?6  T  -f  *?  Trua"  "T1^  c*1,ed  P^US.  FORMAT,  FIND  and 
evaluate.  Emphasized  will  be  a  demonr  .ration  the  use  of  a  FocusFramework 
which  explore,  the  problem  formulation  stage  of  research  through  a  coVper^tTve 
group  activity.  Although  this  instruction  was  designed  to  be  used  with  first-vear 
college  composition  classes,  the  strategy  and  FooT ^amework  have  been  X 
adapted  for  use  in  a  wide  variety  of  other  academic  library  instruction  classes. 
Background  Concerns  of  Library  Instructors 

•  Do  students  have  overall  concept  of  the  research  process? 

•  Does  library  instruction  help  them  to  develop  this  overall  concept? 
Does  library  anxiety  get  in  the  way  of  library  instruction' 

•  Can  an  emphasis  on  problem  formulation  give  student*  a  "jump-start"' 
Does  library  instruction  have  an  effect  on  quality  of  the  research  paper? 

Voice*  of  the  Experts 

•  Kuhlthau's  concept  of  "research  process  orientation" 

•  Mellon  s  concept  of  "library  anxiety" 

I  SS^v*!?0-?6'8  d?«ri?«»n  of  lib-ary  research  as  problem-solving 
NahWakobovits  work  in  domain-integration  in  library  skills  development 

Cognitive  Strategies  Instruction  Designed 

•  First-year  college  composition  course 

•  Three  days;  50-minute  periods 

•  Uses  a  cognitive  framework:  FOCUS,  FORMAT,  FIND,  EVALUATE 

•  Addresses  affective  objectives  at  esch  stage  of  research 

•  Emphasizes  problem  formulation 

•  Incorporates  group  activities  and  cooperative  learning 

•  Aims  at  intermediate  and  long-term  goals 

Research  Study  Designed 

•  Co.-npares  traditional  lecture-based  library  instruction  with 

cognitive  strategies  instruction 

•  Nonequivalent  control  group  design  with  intact  classes 

libraty08itt2udesre8earCh  Pr°CeM  orientation'  ,ibrary  ttnxiety. 

•  Post-instruction  measure  of  topic  and  title  development  (problem  formulation) 

•  End-of-quarter  measure  of  adequacy  of  research  paper  bibliographies 

High  Expectations 

•  More  focused  problem  formulation 

•  Lower  anxiety 

•  More  positive  attitudes 

•  Increased  research  process  orientation 

•  Better  research  papers 
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The  Results 

•  Most  expectations  met 


Summary  of  Performance  Result,  of  Hypotheses  Testing 


Dependent 
Variables 

Research 
>rocess 
Orientation 

library 
Anxiety 

General 
Attitudes 


Pretest 
differences 
Between 
Groups 


nsd 


Research 
Bibliog 


hies 


>osttest 
Differences 
Between 
Groups 


<,0001 


<.01 


nsd 


n/a 


n/a 


n/a 


Gain 

Differences 
Between 
Groups 


<.000l 


nsd 


nsd 


<.0001 


nsd 


<.002 


<.01 


<.05  /  nsd 


n/a 


n/a 


n/a 


Gain 

Within 

Cognitive 

Strategies 

Group 


<.0001 


<.0001 


<.0001 


Gain 
Within 
Traditional 
Group 


<.01 


n/a 


n/a 


n/a 


Hypotheses 
Confirmed 


yes 


nsd 


nsd 


n/a 


n/a 


n/a 


yes 


yes 


yes 


no 


yes  /  no 


Usefulness  of 
Instruction 

Topic/Title 
Development 


Overall,  the  cognitive  strategies  instruction  was  "^^^S^T^ 
strong  performance  of  the  cognitive  strategies  group  on  the  researcn  paper 
biblEvTatS  observed  "-^fttt^ 

cognitive  strategies  group,  but  did  not  «nd  jroup  « ^^^^Sn^an  did  not  detect 
PerCIt^  hoped  that  the  empirical  evidence  of 

process-oriented  instruction  used  as  a  treatment  in  this  th^r Ji"  ^  SS7Se i  to  teach 
£>  design  and  implement  such  «^J^Jft3^ 

the  information  society  itself. 
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Thoughts,  Feelings  and  Actions: 
Integrating  Domains  in  Library  Instruction 

RESEARCH  SUMMARY,  DISCUSSION,  AND  CONCLUSION 

Summary 

acadi*  0ffthi8  *tudy  was  to  comPa"  the  effects  of  two  methods  of 

rvSrW™^0,"  0n  re8earch  proceB8  orientation  (a  construct 
developed  by  Kuhlthau),  library  anxiety  (a  construct  developed  by  Mellon), 
student  performance  on  the  complex  problem-selving  task  of  researching  and 
compiling  a  research  paper  bibliography,  research  paper  topic  and  title 

sessionTlnri  %!£?£0ntuf  Pti10™?**  U8efillneM  °f  the  Kb™?  instructional 
sessions,  and  attitudes  about  library  instruction  in  general. 

Two  instructional  treatments  were  designed  for  use  in  the  librarv  skills 
SKTJ?  of  a  """S^1?"1  En^sh  composition  class.  Bo*  the  tSona 
S3™   ?        COgnitiVe  "trategies  treatment  were  designed  to  support 
SJirT?  °f1?'  "P8  t5rminal  goal' that  of  completion  of  a  successfol, 
fcnSfl.librar:!:iba?ed  re8earch  PaPer-  However,  there  were  considerable 
mfthod  treatments  in  both  instructional  content  and  teaching 

The  traditional  instruction  was  resource-oriented  and  consisted  of  lectures 
fZit™™**?0^*  Pre,ented      ba8ic  to°ls  and  technologies  used  in 
mZSZ  T*"  JS*  tyPu  °?  Plural  instruction  is  common  in  ackdemTc 
library  settings.  The  emphasis  of  the  instruction  was  to  teach  standardized 

wTnd?Le8f°r£Udent  inte7Cti0n  With  the  libraiy  environment   N em^hlis 
SSLft      °n    6  §r0Ce88  °/  re8earch»  except  for  a  recommendation  to  start  ™th 

ho«,!  ^^F^6.  8tratee*e8  instruction  was  process-oriented,  that  is,  it  was 

ut  „f  .n^em/ha8,fed  the  research  proce88  iteelf  rather  than  focusing  oTJhe 
use  of  specific  information  sources.  The  design  of  this  instruction  waV  based  on  a 

iS^n1nframTrk  Which  COmh™i  co"ceptual  mJS^SSS^  8 

searching  as  a  type  of  problem-solving  with  grounded  theories 
JIKE  g      °?Va  andJme*acognitive  aspects  of  information  searching  The 
Knt  J™*-6818?6,?    inte^,ato  developnient  of  skills  in  all  the  domains  of 
snSSMEF'"  ,kl1  "  f°r  Problem-8olvin&  Psychomotor  skills  for  navigating  in 
StK^Sr^  m8t^itiv8         *r  self-monitoring^* 

raauirsdEnS^ei,«.for       8t?dy  T  190  un°ergraduate  students  enrolled  in  a 
required  English  composition  class.  The  research  design  chosen  for  tlvs  studv  was 
a  non-equivalent  control  group  design  with  intact  classVs  randomly  assigned to  the 
cognitive  strategies  group  or  the  traditional  group.  The  researcher  an 
experienced  library  instructor,  taught  .11  ..cfion.'u.ing  a  leSureand^discussion 

studv  lh™fdiTereilt.tJ?e8  °! ;,va,ua«on  instruments  were  employed  in  this 
study.  The  pretest,  administered  at  the  beginning  of  the  quarter  included  a 
survey  which  collected  background  demographic  data  Th ^/eteVt X  indud-d 

mZSZ^A*?"  "nd  ,iburaIy  anx,ety-  I"»m-dl«toly  after  instruction,  held  mid- 
quarter,  students  were  asked  open-ended  questions  (on  topic  and  research ^ 

ffimme.d°ff  ,nt  ft"fd  8UrV/ Wl0n,1wn,ch  tn-ured  student  p  cep  ion.  of 
SmSj^d  -^2ftlln--,i?uthB  HbM,ry  ln8tru^on  sessions.  The  ?osttest, 

i^S^AS^lSiS^  qU*?T'  TT*t9d      r,8"aroh  PwS 'orientation, 
general  attitudes,  and  library  anxiety  scales.  FJna  y.  students'  research  nanar 
bibliographies  were  scored  at  the  en«f  of  the  quarter  & fif! "i.,, ^hfeh  msaSred 


1172 


adequacy  of  student  performance.  Copies  of  these  instruments  and  protocols  for 

their  administration  can  be  requested  <Tom  the  researcher. 

their  adm,mstra«  ^         ^  gtrategie8  treatment  would  have  more 

positive  effecto  than  the  traditional  treatment  on  research  paper  topic  and  title 
development,  and  on  student  perceptions  of  the  uwfiihe.s  of  the  library 
instructional  sessions  as  observed  immediately  after  ins truction    Jt  was  also 
exnected  that  the  cognitive  strategies  treatment  would  result  in  higher  student 
g  Jns  over  the  coursfof  the  academic  term  in  research  process  onentotoon, 
Sudes  about  library  instruction  in  general,  anxiety  reduction,  and  performance 

"  "finlt  samp°lfahsyu  were  conducted  to  determine  differences  between 
treatment  groups  on  the  research  process  orientation  library  «* 
crVneral  attitudes  measures  of  both  the  pretest  and  posttest  Additional 
fXendeTsampTetteste  compared  gain  score,  between  treatmen •W™%™ 
each  of  these  measures.   Paired  sample  1-test.  were  conducted  on  each  portionof 
the  pretest  and  posttest  to  determine  the  significance  of  gains  in  research process 
onenteE  Hbrary  anxiety  and  general  attitudes  within  each  treatment  group. 
The  £tt  of  sSuent  performance  on  the  research  paper  bibliographies  were  also 
tested K  an Independent  sample  1-test,  as  was  the  student  perception  of  th 
uifless  of  instruction.   The  categorical  date  produced by  ^SSfeSSL 
topic  and  title  development  was  analyzed  using  chi-square  to  determine  differences 
between  the  groups^  ^  ^  Background  gurvey  dat         ^  no 

sienificant  differences  between  the  treatment  groups  on  »ucnldeinoP»Ph'c/f3" 
as^wrace  sex,  academic  status  and  grade  point  average   Independent  sample 
"teste'eonducted  on  the  pretest  data  showed  no  difference,  between  the  group,  in 
research  proce..  orientation  or  general  attitude..  There  was  however,  evidence 
Tat  the  SlaV  poup  demonstrated  significantly  lower  library  anxiety  than 

the  M«altTe?o^  was  successful  in  improving 

research  process  orientation,  reducing  library  anxiety,  and  mf™«"S  general 
attitudes  Evidence  for  the  effectiveness  of  this  treatment  was  also  shown  by 
Valuator  judgements  of  more  focused  re.earch  paper  topfcs  and  tatie.  among  the 
cognitive  strategies  group  and  especially,  in  the  '^SIU!!SS!r^^!Ln  of 
cognitive  strategies  group  on  the  research  paper  bibliographies.  The  measure  ot 
peSons  of  immediate  usefulness  of  library  instruction  did  not  detec *  any 
Significant  group  differences  with  both  groups  indicating  strongly  positive 
perceptions. 

Admission  *f  Findings 

^Sagtdy  do  support  the  hypothesis  that  research  process 
orientation  (RPO)  for  both  treatment  groups  would  ^Vl^tl^Z  measure 
pretest  measure  at  the  beginning  of  the  academic  Quarter  to  the V™"?*™"™ 
at  the  end  of  the  quarter.  The  data  also  clearly  indicate  that  ^e  cognitive 
Strategic  g^oup  gained  in  RPO  to  a  significantly  greater  extent  than  did  the 
traditional  group^     ^        ^  ^  ^  beginning  of  the 

quarter  to  the  end  was  to  be  expected  simply  due  to  the  ^S^S^ulff*  * 
research  paper  project  from  start  to  finish.  Kuhlthaus  studies  (1988a,  1988b, 
Kuhlthau  et  al.,  1990),  which  measured  research  process  orientation  at  the 
Sning,  middle  and  end  of  the  research  paper  project,  found  an  increasing  level 
^RPo"hr^ghout  the  project.  Kuhlthau's  subjects,  however  did  not  receive  any 
kind  of  library  instruction  during  the  project.  The  present  study  is  unique  m 

measuring  RPO  in  relation  to  library  instruction.   iMt-A 

The  finding  that  library  instruction  which  was  ^^fSSSST^ 
and  domain-integrated  was  more  effective  in  increasing  the  RPO  of  students  is 
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!iS!?CfntV*Jhe  emPh*8i8  fa  the  instruction  on  the  affective  and  metacognitive 

FOCUS ^rote^mV'T^J^^6^/ 41,6  Btrate^,  * 

avS{  c, ^!T!1FIND™nd  EVALUATE,  seemed  clearly  effective  in  increasine 

82tBR-f  ?"  8tU»le^  ?,B  ?nding  8upport8  a  body  of  research  wWchTnTcates 
^rrf&  MuX  ^™?el£?er  ?°W  *°  ?eam"  throu*h  di"ct  in.trucironCate8 
« rSSLi/S hy'.1986)'  W1?116  direct  experience  of  the  research  process  is  likely 
fnS££  ""Rasing  research  process  orientation,  the  findings  in  this  study  * 
5£S£™ at  Procass-onente d  instruction  may  influenca  the  development  of  RPO. 
^^LWu0£r0V1ie?*truCiT  for  beginning  researchers  should  be 
encouraged  by  these  findings  to  design  instruction  which  presents  research  as  a 

researchers  to  explore  this  issue  farther.  The  present  study  looked  at  research 
process  orientation  as  one  of  six  dependent  variables,  and  based  measurement  rf  it 
Zm^VX^J.  ISK  ^  8tudie8  8h,0uld  ^ttor SatoXTriabl  °f  * 

.f  A'  a™*  re8earch  P,roce»»  a«  experienced  by  undergraduate  students  is  beintr 
SSSh™  I  T*?  ?l™yB'  ^°th  <>uahtative  ««d  quantitative.  Fistor  (in  pS 
^«^UiCtod  ,n'd?t1h  inte™ew8  with  students  who  were  identified  as  haSng 
So^«f?ily.?i!nPlet5d  re8earch  project8  of  variou8  kinds.  Her  initial  finding? 
sugp^st  that  the  students'  experiences  in  finding  a  focus,  using  evide^nc"  anf 

E«E£!S* different thL  the  strategie s l££££»fct 
m  library  instruction.  Content  analysis  of  oral  or  written  libra™  ^er  I^mSE 
are  being  used  by  Nahl-Jakobovits  and  Jakobovits TKS)  toTCE^KZS 
objective,  in  the  affect  ve,  cognitive,  and  sensorimotor  domain.  OutoXth^TfiVW 

iSS?  *  ft""6"'  Sarl<?  and  Pa]m^  (WW  are  using  ComputerLand  tools  U, 
analyze  written  and  spoken  texts  for  the  purpose  of  representing  menteTmodels  of 

Sf-nil1"  8^0Clal  ?tuati0n8-  Using  this  technique,  the  re8earchefsTav?ex™r«ted 
«!^LTn°del8  °f  ""dergraduate  students  at  the  beginning  and  end  points  of  a 
research  paper  assignment  and  compared  the  coimTuter-fenerate ma£s 

work  don bv ™{X\™$?BAhr  ie8Crib6d  above'  ™mbined  with  "he  basic 
worK  done  by  Kuhlthau,  could  lead  to  a  deeper  understanding  of  how  the  re«.««* 
process  ,s  carried  out  by  individuals,  and  how  it  changTove?  Sim 'art  12 T 
experience  Such  an  understanding  would  be  valuablf  not  only  for  Hbrarians  and 
hbrary  instructors,  but  for  designers  of  instructional  products   More  and  ™r« 
reference  books  are  being  published  in  electron! ^  fomate  such  as^lTO*?*. 
information  is  available  on  electronic  databases  everyday.  The. ^wurfes  witt. 
their  tremendous  potential  for  interactive  searching, Xu  d  be  dSed  Sith 
human  information  searching  process  clearly  in  mind  designed  with  the 

Lihrnrv  Anv^fy 

both  tr«2,™Ie8Ult8  °f  thi8  ¥?iy  do  8UPport  ^e  hypothesis  that  library  anxiety  for 
both  treatment  groups  would  decrease  from  the  pretest  measure  at  the  ErimL. 
of  the  academ  c  quarter  to  the  posttest  measure  aUh Tend Tthe  quarter  g 

oS7&  *   a  ?*  d,eCrease  for  the  'W*™  't^tegies  poup  ITsiScant 
Oat  of  the  traditional  group  was  not.  The  data  indicate  th  a -libra  w  KtTin  th« 
cognitive  strategies  group  was  reduced  to  a  significantly  weatol^e.  S/n  ?n 

tSSStS^t  nS68e  fl^ng8  8UpP°rt  ™"ch  wffit,^wl?hantm 
•ru^f  «*—  "  the 
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Although  the  library 
with  a  Cronbach  Alpha  Coefficient  «  . 8  N  « ^J^t  &on  (1986)  had 
deal  of  variability  in  student  ^^^^^L'S^ng  petition  of  self 
identified  several  probable  sources '  ^^Jg^^f ti^,  fear  of 
as  inadequate,  perception  of  othert  9\^°J  Jne^S  discomfort  in  unfamiliar 
approaching ^mediators  and ^[^X^  ZT^n  to  reflect  these  . 

h^me  allfn  their  perceptions  of 

anxiety-producing  situations.  exo'anation.  It  is  likely  that  the 

Tbl.  obMMtton  W^TSxSZ  M^^d^MnMit.  Although  the 
library  anxiety  scale  itself  n^\^^^^S^  «nd  empirical  evidence,  the 
content  of  the  items  is  based  m*^£^SCnnat  been  validated  by 


individual* 
of  the  problem 


^™^8  ft  fotSs^s^^^ 

gupports  using  an  instructions  X°n inftraSn  about  the 

research  process,  and,  while  doing  "^^"^^s 3»ut  libraries  are 
library  building,  technologies,  and  ^^^.^J^ *  developed  over  years 
difficult  to  change,  especially  since  such  "'J^"  h"J" Tin Torder  to  change 
of  schooling  and  library  use.  'f^^.t^^S^^S,£  be  met.  The 
attitudes,  both  the  *f^^ffl^SS^^«S^!^  of  the  object, 
internal  conditions  demand  that  the  «dnt^ nave  a ™    *    .   .    h  case  of 

event,  or  person  to  ^J^^^9^^^^^^!*  ol\«*s 
negative  attitudes  towards  libraries .  it ^can  ^"^^jX^Lg itself,  the 
toward  which  those  attitudes  are  greeted  They  11 nd '^ff™  °umoreB  But  thege 
multiple  technologies  contained  ^^^L^iTtSSl  and  situations  of 
negative  attitudes  are  also  directed  ^^^St^Spi^^  ™*  have 

ncept  to  most  the  small 

numbe'o^  one 
JT=fflftS3  ^Classes 
which  teach  the  library  research  f^J^WSX;  participated  in,  the  49% 

WfSE™ SotUriring.  Ubrary 
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instruction  has  a  common  reputation  for  naLbeing  interesting.  That  onlv  42%  of 
students  gave  those  answers  on  the  posttest  was  only  slighSy  heartnmg 

Prflblfim-Sftlving  Performance-  Th*  Baagarfih  Pap«r  RihHnr^hv 
hvnnthJ^a  ™PPrtant  outcome  of  this  study  was  the  strong  support  for  the 

^at  concerns  student  performance  on  the  profiem-sol"™  talk  of 
resewching  and  compiling  a  research  paper  bibliography   Result?  of  2i  .Shv 

Wilson,  1984)  and  student  research  products  (Kohl  &  Wifion,  1986? 

msmm 

i^stiijizr u  evalu"'4  *•  5tade"t  u 

Ct a tTZfc K?        *"»■'»"•.  *•  'valuator.  eJSd 
Perceptions  of  LEaflfiilnaia  atLifaaa  Tnfltmrtr" 

BtrategieTgr^^^  the  hypothesis  that  the  cognitive 
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papers.  On  another  question,  only  five  percent  of  the  student* ;  found  *e  "8810n» 
to  be  a  waste  of  time.  Responses  were  similar  for  questions  which  dealt  with 
specific  steps  of  the  research  process.  Although  the  traditional  K£up  idnot 
receive  specific  instruction  in  the  steps  of  the  research  process,  they  aPPa™™*  ,n 
did  feel  that  the  more  reBource-oriented  instruction  they  received  prepared  them  to 
undertake  the  research  process.  „ 

A  ceiling  effect  on  the  scores  may  be  responsible  for  the  fa^ure  tofind 
differences  between  the  groups.  Ninety  percent  or  more  of  all  subjects  responded 
positively  or  strongly  positively  on  every  usefulness  item  Anotiier  explanation  for 
S  faitoe  to  find  differences  may  be  that,  according  to  the  student  perception., 
the  treatments  were  equally  beneficial  in  preparing  them  to  proceed  with ,  the 
research  paper  assignment.  This  student  perception  is  supported  by  informal 
comments  t£  the  course  instructors.  All  three  of  the  English  professors  who  had 
SrTlasses  participating  (one  in  each  treatment  group)  sat  in  on  both  treatments 
and  were  uniformly  pleased  with  both  class  sessions. 

R*gftflrr.h  Pappr  Tn^r.  and  Title  Development  ,  , 

The  prediction  that  the  treatment  groups  would  differ  in  the  level  ot 
development  of  the  research  paper  topic  and  research  paper  title  immediately 
after  instruction  was  supported  by  the  results  of  this  study.  As  expected,  the 
cognitive  strategies  group  was  more  frequently  judged  to  have  narrowed  or 
focused  topics  and  titles  than  the  traditional  group. 

These  findings  were  very  likely  the  result  of  the  problem  formulation 
strategy  presented  to  the  cognitive  strategies  group  on  the  first  day  of  instruction. 
Students  in  this  group  had  had  group  practice  in  using  a  framing  strategy  to 
select  and  narrow  their  research  paper  topics,  and  had  been  given  copies  of  the 
strategy  worksheet  (available  from  the  researcher)  for  their  own  personal  use. 
Students  in  the  traditional  group  had  been  encouraged  to  narrow  and  focus  their 

topics,  but  had  not  had  any  specific  instruction  on  how  to  do  so.  Library   

instruction  which  is  closely  integrated  with  assignments  has  been  found  to  be  more 
effective  than  general  instruction  (Oberman,  1984). 

There  was  some  threat  to  the  internal  validity  of  this  measure  due  to 
history  bias.  Due  to  timing  and  due  date  differences  in  various  professors 
assignments,  some  students  may  have  felt  a  strong  necessity  to  narrow  their 
topics  and  generate  their  term  paper  titles  by  the  last  day  of  library  instruction 
while  others  were  on  a  less  immediate  time  schedule.  Future  researchers  should 
control  for  that  threat  by  negotiating  uniform  assignments  and  due  dates  with  the 
professors. 

(rfinfirnl  P'smisflinn 

Overall,  the  results  of  this  study  support  the  use  of  a  domain-integrated 
process-oriented  approach  to  academic  library  instruction.  The  cognitive 
strategies  group  showed  greater  improvement  in  research  process  orientation, 
more  reduction  in  library  anxiety,  and  more  positive  general  attitudes  about 
library  instruction  than  did  the  traditional  group.  Moreover,  the  cognitive 
strategies  group  demonstrated  more  developed  research  paper  topics  and  titles, 
and,  mcst  importantly,  were  judged  to  have  compiled  more  adequate  reMarch 
paper  bibliographies  than  the  traditional  group.  Student  perceptions  of  both 
methods  of  instruction  were  strongly  positive.  .,   .....  . 

Certain  limitations  of  the  study  may  undermine  the  generaluability  ot 
these  positive  findings.  The  investigation  was  limited  te  a  single  field  experiment 
conducted  at  one  institution.  One  individual  taught  all  the  library  instruction 
sections,  thus  making  it  difficult  to  establish  that  the  results  could  be  applied  to 
other  people  in  other  times  and  places.  Other  studies  that  evaluate  instruction 
based  on  the  cognitive  strategies  treatment  method  may  help  in  providing 
evidence  for  the  external  validity  of  this  study.  ... 

The  fact  that  only  one  person  taught  the  classes  also  may  bring  into 
question  the  internal  validity  of  the  study.  The  researcher  administered  both  of 
the  treatments,  thus  acting  as  both  investigator  and  teacher.   There  were 
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practical  considerations  which  led  to  the  decision  to  administer  the  treatments  in 
this  way.  The  treatment  was  long,  consisting  of  three  hours  of  instruction  on  three 
consecutive  days.  Due  to  job  assignments,  it  was  not  possible  to  schedule  another 
librarian  for  the  twenty-four  hours  of  instruction  necessary  for  the  eight  intact 
classes.  Consideration  was  given  to  training  several  other  librarians  to  share  in 
administering  the  treatments.  This  option  appeared  to  be  unworkable  due  to  the 
training  time  required  and  the  likelihood  of  introducing  teacher  effect,  a  threat  to 
internal  validity. 

Experts  in  experimental  research  strongly  discourage  the  dual  role  of 
"Y"fa*at°r  *"d  teacher  due  to  the  inherent  danger  of  experimenter  bias  (Barber, 
1973).  This  bias  can  be  introduced  in  several  ways  including  failure  to  follow 
protocol.  If  the  experimenter  deviates  significantly  from  the  specified  treatment, 
the  study  that  is  actually  conducted  may  not  be  the  same  as  the  one  that  is 
reported.  Another  way  this  bias  can  be  introduced  is  through  the  expectancy 
effect  Experimenters  (and  teachers)  commonly  expect  certain  results;  they  expect 
that  one  group  will  perform  differently  from  the  other.  This  expectancy  may  result 
in  an  unconscious  and  unintentional  influence  which  may  affect  the  outcome. 

To  minimize  these  problems,  care  was  taken  to  follow  the  instructional 
treatments  precisely  and  uniformly.  Evidence  that  this  was  done  can  be  seen  in 
the  similarity  of  student  perceptions  of  the  usefulness  of  library  instruction  as 
measured  immediately  after  instruction.  Further  evidence  can  be  provided  by  the 
English  professors  who  sat  in  on  the  instructional  sessions.  In  informal 
observations,  all  those  who  had  sections  taught  by  both  methods  found  the 
sessions  to  be  equal  in  quality.  A  formal  observation  checklist,  including  such 
criteria  as  accuracy,  interaction,  friendliness,  comprehensiveness,  etc,,  would 
have  been  useful  in  documenting  these  observations. 

.  a  jTa  j*?!  ini°  account  ,the  limitations  outlined  above,  the  findings  of  this 
study  do  add  to  the  relatively  small  body  of  empirical  evidence  concerning  the 
effectiveness  of 'process-oriented  instruction.  Studies  of  ongoing  library  instruction 
programs  based  on  the  process  approach  (Lynch,  1989;  Mellon,  1984;  Pask  & 
Smith,  1989)  are  expected  to  increase  this  body  of  evidence.  More  studies  are 
needed,  not  only  to  evaluate  the  effectiveness  of  programs  on  students'  library 
[XmatiSn  skills111  *°  eXamine  the  wider  81,(1  mor*  lonK"range  effects  on  lifelong 

Practically  speaking,  the  cognitive  strategies  treatment  has  been 
nnplemented  as  the  method  of  library  instruction  used  at  Valdosta  State  College 
Other  librarians  have  learned  the  method  and  have  improved  upon  it;  oneoine 
program  evaluation  ma/  prove  to  add  useful  information  in  the  future. 

«  Conclusion 
The  basic  concern  which  led  to  this  study  was  a  desire  to  make  library 
instruction  more  appropriate  to  the  needs  of  users  today  and  in  the  future. 
Information  is  today  considered  to  be  the  basis  of  the  economic,  social,  and  political 
structure  of  our  society.  In  this  society,  people  must  have  self-directing 
en^amentTanaeement  Ski"8  Whi°h  "*  adaptive  to  cha"Png  technologies  and 

t,  4  .   A  rev?ew  of  the  literature  of  library  and  information  science,  as  well  as 
that  in  learning  theory,  suggested  that  the  skills  needed  by  a  self-reliant 

mrr^^n«|ep*er,8if±^  t0Jh,T  in?olved  in  Problem-solving  (Dervin, 
J2L  • a«n<J'.19?5i  Reichel,  1987).  Field  studies  by  Kuhlthau  (1983,  1988a, 

JnSnUA^W      i  ?rocess  °f  lib«"-y  research,       P™vided  data  which 
support  the  problem-solving  model  and  led  to  the  development  and  validation  of  a 
new  model  of  the  research  process  (Kuhlthau,  Truock,  George  &  Belvin,  1990). 
I  his  model  included  not  only  the  actions  taken  by  researchers,  but  their 
corresponding  thoughts  and  feelings  as  well.  Kuhlthau's  work  provided  the 
SKSL of  ""earcn.p.roce»»  orientation,  one  of  the  dependent  variables  used  in 
this  study.  The  cognitive  strategies  method  of  library  instruction  was  shown  to 
increase  research  process  orientation  in  the  context  of  this  study.  If  this 
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imorovement  is  repeated  over  a  broad  range  of  conditions  and  instruction,  this 
cZld  be  wen  as  a  step  toward  the  goal  of  developing  self-reliant  information 
managers  (Lechner,  1989).  .  „„.  u_ 

A  number  of  researchers  have  suggested  that  anxiety  is  a  major  problem 
which  must  be  overcome  in  order  for  library  users  to  be  dale  ^  Jake  Ml 
advantage  of  library  instruction  Rnd  library  use  (Kuhlthau  et  al.,  1990;  Mellon, 
1986).  Mellon  (1986)  documented  students'  attitudes  about  libraries  and  the 
search  process,  and  identified  probable  sources  of  fears  and  anxiety.  Hot  work 
was  important  in  developing  the  construct  of  library  anxiety,  used  as  a  dependent 
v^riaHeinthisst       "file  cognitive  strategies  method  f  f ™Mf™£"WM 
shown  to  decrease  library  anxiety  in  the  context  of  this  study.  If  this  reduction 
occurs  repeatedly  in  other  contexts,  this  too  may  support  the  fal  of  self-reliance 
in  information  searching  (Lechner,  1989;  Nahl-Jakobovits  &  Jakobovits,  1986). 

The  other  dependent  variables  of  general  attitudes  about  library 
instruction,  perceptions  of  usefulness  of  library  instruction,  research  topic  and  title 
development  and  performance  on  the  research  paper  bibliography  measured  the 
more  immediate  goals  of  library  instruction.  Student  achievement  of  these 
immediate  goals  provides  evidence  of  movement  toward  the  achievement  of  long- 
term  coals  such  as  self-reliance  in  information  searching  (Lechner,  1983). 

It  is  hoped  that  the  empirical  evidence  of  the  effectiveness  ot  the  domain- 
inteerated  process-oriented  instruction  used  as  a  treatment  in  this  study  will 
encourage  librarians  to  design  and  implement  such  instruction.   Libraries  should 
make  it  their  role  to  teach  not  only  specific  sources  and  procedures  within  the  local 
library,  but  to  teach  the  processes  of  life-long  self-education  in  a  worldwide, 
electronic  environment.  In  this  global  view,  the  library  is  no  longer  the  context  for 
the  learning  process;  instead,  the  context  is  the  information  society  itseit. 
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